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PREFACE 

This  Sixteenth  Edition  (version  1 .0)  of  the  DAU  Program  Managers  Tool 
K/f  contains  a  graphic  summary  of  acquisition  policies  and  manage¬ 
rial  skills  frequently  required  by  DoD  program  managers.  It  is  a  current 
version  of  a  “Tool  Box”  that  was  first  developed  by  Charles  F.  Schied 
of  the  Defense  Acquisition  University  (DAU)  Program  Management 
Course  (PMC)  92-1 .  The  information  in  the  Tool  Kit  is  extracted  from 
DAU  course  material  and  is  based  on  DoDD  5000.01  (May  12,  2003), 
DoDI  5000.02  (December  8,  2008),  the  Defense  Acquisition  Guide¬ 
book  (DAG),  (August  5,  2010),  CJCSI  6212.01  E  (December  15,  2008), 
CJCSI  31 70.01  G  (March  1 , 2009),  the  Joint  Capabilities  Integration 
and  Development  System  Manual  (July  31 , 2009),  and  the  Weapon 
Systems  Acquisition  Reform  Act  of 2009.  Material  from  the  DAU  Acker 
Library  and  Knowledge  Repository  was  also  used. 

Since  the  DAU  Program  Managers  Tool  Kit  is  a  compilation  of  class¬ 
room  presentation  and  teaching  materials  used  in  a  number  of  different 
courses  at  DAU,  the  charts  and  tables  vary  in  look  and  feel. 

Users  of  the  Tool  Kit  are  reminded  that  this  summary  is  a  guide  only 
and  should  not  be  used  as  a  substitute  for  official  policy  guidance.  Peri¬ 
odic  review  of  official  policy  guidance  is  recommended. 

An  e-Toolkit  has  been  developed  which  contains  current  updated  ac¬ 
quisition  guidance  and  direction  (https://pmtoolkit.dau.mil).  This  (“hard¬ 
copy”)  Toot  Kit  is  current  as  of  January  1 , 201 1  and  is  extracted  from 
the  e-Tool  Kit.  The  hard  copy  Tool  K/fcan  be  found  at  http://www.dau. 
mil/pubscats/pages/tool%20kit.aspx. 
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CHAPTER  1 

ACQUISITION  PROCESSES  AND  RESOURCES 


•  Things  that  make  you  go  “Hmmm?...” 

✓  The  only  thing  most  auditors  fix  is  the  blame. 

✓  Experience  is  something  you  got  just  after  you  needed  it. 

✓  People  are  smarter  than  they  look;  listen  to  them. 

✓  The  last  10  percent  of  the  performance  sought  generates  one- 
third  of  the  cost  and  two-thirds  of  the  problems. 

✓  Never  open  a  can  of  worms  unless  you  want  to  go  fishing. 

✓  Those  who  believe  it  cannot  be  done,  will  you  please  get  out  of 
the  way  of  those  who  are  busy  doing  it? 

•  Things  we  shouid  aiways  remember. 

✓  Be  honest  in  everything  you  say,  write,  and  do. 

✓  Be  good  to  your  people,  and  they  will  be  good  to  you. 

✓  Forgiveness  is  easier  to  obtain  than  permission. 

✓  Keep  everyone  informed;  when  in  doubt,  coordinate. 

✓  Be  the  first  to  deliver  bad  news. 

✓  Bad  news  does  not  get  any  better  with  time. 

✓  If  you  are  sitting  at  your  desk,  you  are  not  managing  your 
program. 
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THE  PROGRAM  MANAGER’S  BILL  OF  RIGHTS 
AND  RESPONSIBILITIES 


Rights: 

Program  Managers  have  the  RIGHT  to: 

•  a  single,  clear  line  of  authority  from  the  Defense  Acquisition 
Executive; 

•  authority  commensurate  with  their  responsibilities; 

•  timely  senior  leadership  decisions; 

•  be  candid  and  forthcoming  without  fear  of  personal  consequences; 

•  speak  for  their  program  and  have  their  judgments  respected; 

•  receive  the  best  available  training  and  experience  for  the  job;  and 

•  be  given  adequate  financial  and  personnel  resources. 

Responsibilities: 

Program  Managers  have  the  RESPONSIBILITY  to: 

•  accept  program  direction  from  acquisition  executives  and  implement 
it  expeditiously  and  conscientiously; 

•  manage  their  programs  to  the  best  of  their  abilities  within  approved 
resources; 

•  be  customer-focused  and  provide  the  user  with  the  best,  most  cost- 
effective  systems  or  capabilities; 

•  innovate,  strive  for  optimal  solutions,  seek  better  ways  to  manage, 
and  provide  lessons-learned  to  those  who  follow; 

•  be  candid  about  program  status,  including  risks  and  problems  as 
well  as  potential  solutions  and  likely  outcomes; 

•  prepare  thorough  estimates  of  financial  and  personnel  resources 
that  will  be  required  to  manage  the  program;  and 

•  identify  weaknesses  in  the  acquisition  process  and  propose 
solutions. 
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LEGEND: 

CDR— Critical  Design  Review  lOT&E— Initial  Operational  Test  &  Evaluation  SCMPD— System  Capability  &  Manufacturing 

FOC— Full  Operational  Capability  ISD— Integrated  System  Design  Process  Demonstration 

FRF— Full-Rate  Production  LRIP— Low-Rate  Initial  Production 

IOC— Initial  Operational  Capability  FDR— Preliminary  Design  Review 
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ACQUISITION  CATEGORIES  (ACAT) 


Major  ACAT  ID: 

Defense 

Acquisition 

Programs 

ACAT  iC: 


•  DAB  Review 

•  Designated  by  USD(AT&L) 

•  Decision  by  USD(AT&L) 

•  Component  Review 

•  Designated  by  USD(AT&L) 

•  Decision  by  Component  He 


$365M  RDT&E  or$2.190B 

Procurement 

(FY  00  Constant  $) 


Major  AIS 

Acquisition 

Programs 

ACAT  lAM: 

ACAT  lAC: 

•  ITAB  Review 

•  Designated  by  USD(AT&L)* 

•  Decision  by  USD(AT&L)* 

•  Component  Review 

•  Designated  by  USD(AT&L)* 

•  Decision  by  Component  Head/CAE 

$378M  Life  Cycle  Cost  or 
$126M  Total  Program  Cost 
or  $32M  Program  Cost  in 
any  single  year 
(FY  00  Constant  $) 

Major 

Systems 

ACAT  II: 

•  Does  Not  Meet  ACAT  I  Criteria 

•  Designated  by  Svc  Sec/CAE 

•  Decision  by  Svc  Sec/CAE 

$140M  RDT&E  or 
$660M  Procurement 
(FY  00  Constant  $) 

Aii  Others 

(except 

Navy  and 
USMC) 

ACAT  III: 

•  Does  Not  Meet  ACAT  I,  lA,  or  II 
Criteria 

No  Fiscal  Criteria 

•  Designated  lAW  Component 

Policy 

•  Decision  lAW  Component  Policy 

Navy 

USMC 

ACAT  IV: 

•  Not  otherwise  designated  ACAT  I, 
lA,  II,  or  III 

•  Designated  lAW  Navy  Policy 
•Navy/USMCACATIVT/IVM 

•  Decision  at  lowest  appropriate  level 

SECNAVINST  5000.2_ 

LEGEND: 

ACAT— Acquisition  Category  iTAB—  information  Technoiogy  Acquisition  Board 

AiS— Automated  information  System  RDT&E—  Research,  Development,  Test,  and  Evaluation 

CAE—  Component  Acquisition  Executive  USD(AT&L)—  Under  Secretary  of  Defense 

DAB—  Defense  Acquisition  Board  (Acquisition,  Technoiogy  and  Logistics) 

*  May  be  deiegated 
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ACQUISITION  STRATEGY  CONSIDERATIONS 

{Defense  Acquisition  Guidebook,  Chapter  2) 


•  Acquisition  Approach 

—  Modular  Open  Systems  Approach 
Summary 

—  Tailoring 

•  Source  &  Related  Documents 

•  Capability  Needs 

•  Top-Level  Integrated  Schedule 

—  EMD  Top-Level  Schedule  * 

—  MS  C  &  FRP  Top-Level  Schedule  * 

•  Program  Interdependency  &  Interoperability 
Summary 

•  International  Cooperation 

—  Test  Requirements  for  Export  of 
Defense  Systems 

•  Risk  &  Risk  Management 

•  Technology  Maturation 

•  Industrial  Capability  &  Manufacturing 
Readiness 

—  Industrial  Capability 

—  Elevating  Industrial  Capability  Issues 

—  Industrial  &  Manufacturing  Readiness 

—  Sustaining  Industrial  Capabilities 

•  Business  Strategy 

—  Small  Business  &  Small  Business 
Innovation  Research 
♦Subcontracting  Plan/Small  Business 

Participation 

♦  Performance  Measurement 

♦  Small  Business  Innovation  Research 
Considerations 

—  Contract  Approach 

♦  Performance-Based  Business 
Strategy 

-  Modular  Contracting 

♦  Contracting  Bundling  or  Consolidation 

♦  Major  Contract(s)  Planned 

-  Contract  Type  Selection 

-  Sustainment  Procurement  Strategy 

♦  Multi-Year  Contracting 

♦  Contract  Incentives 

♦  Warranties 


♦  Leasing 

♦  Developmental  Testing  Requirements 

♦  Incorporation  of  Systems  Engineering 
Requirements 

•  Market  Research 

•  Competition 

•  Resource  Management 

—  PM  Office  Staffing  &  Support 
Contractors 

—  Cost  &  Funding 

—  Cost  Control  &  CAIV  Plan 

—  Earned  Value  Management 
—Advanced  Procurement 

•  Program  Security  Considerations 

—  Information  Assurance 

—  Critical  Program  Information  &  Program 
Protection  Plan  Summary 

-Anti-Tamper  Measures 

—  Supply  Chain  Risk  Management  Key 
Practices 

•  Test  and  Evaluation 

•  Data  Management 

—  Data  Management  &  Technical  Data 
Rights 

—  Integrated  Data  Management 

•  Life-Cycle  Sustainment  Planning 

—  LCSP  Executive  Summary  for 
Acquisition  Strategy 

•  Life-Cycle  Signature  Support  Plan 

•  Chemical,  Biological,  Radiological  and 
Nuclear  Survivability 

•  Human  Resources  Integration 

•  Environment,  Safety  and  Occupational 
Health  (ESOH) 

•  Military  Equipment  Valuation  & 
Accountability 

—  Proper  Financial  Accounting  Treatment 
for  Military  Equipment 

—  Accounting  Review 

•  Corrosion  Prevention  &  Control  Plan/ 
Strategy 


NOTES:  1)  The  content  of  the  Technology  Development  Strategy  (TDS)  due  at  Milestone  A  is  very  similar 
to  the  Acquisition  Strategy  due  at  Milestone  B.  2)  In  addition  to  the  Acquisition  Strategy,  there  are  tive  plans 
required:  Acquisition  Plan  (FAR/DFARS),  Program  Protection  Plan  (PPP),  Test  and  Evaluation  Master  Plan 
(TEMP)  (DoDI  5000.02),  Information  Support  Plan  (ISP)  (CJCSI 621 2.0t  E ),  and  Systems  Engineering  Plan 
(DoDI  5000.02). 
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ACQUISITION,  TECHNOLOGY  AND  LOGISTICS 
(AT&L)  KNOWLEDGE  MANAGEMENT  SYSTEM 

<  https://acc.dau. mil/at&lkm  > 

(Composed  of  the  following  subsystems) 

Defense  Acquisition  Portal  (DAP) 

The  Defense  Acquisition  Portal  (DAP)  <https://dap.dau.mil/> 
was  launched  to  replace  the  AT&L  (Acquisition,  Technology,  and 
Logistics)  Knowledge  Sharing  System  (AKSS).  Like  its  predeces¬ 
sor,  DAP  continues  to  provide  acquisition  information  for  all  DoD  service 
components  and  across  all  functional  disciplines.  DAP  serves  as  the  central 
point  of  access  for  all  AT&L  resources  and  information,  and  to  communicate 
acquisition  reform.  As  the  primary  reference  tool  for  the  Defense  Acquisi¬ 
tion  workforce,  it  provides  a  means  to  link  together  information  and  refer¬ 
ence  assets  from  various  disciplines  into  an  integrated,  but  decentralized, 
information  source.  Information  is  organized  under  eight  major  tabs:  Home, 
Acquisition  Process,  Workforce,  Policy,  Communities  of  Practice,  Training 
and  Education,  Industry,  and  DAU.  DAP  is  one  part  of  the  AT&L  Knowledge 
Management  System  (AKMS). 

A  major  feature  in  DAP  is  the  Acquisition  Career  Gateways  <https://dap. 
dau.mil/career/Pages/Default.aspx>,  where  each  acquisition  career  field 
provides  specific  career,  training,  and  job-specific  information  and  resourc¬ 
es.  To  tie  it  all  together,  the  Search  feature  has  been  reengineered  to  make 
it  easier  to  find  specific  learning  assets  or  job  support  tools  in  the  Portal. 

Ask  A  Professor  (AAP) 

Ask  a  Professor  (AAP)  <https://akss.dau.mil/aap>  is  a  service 
offered  as  part  of  AKMS.  Users  submit  acquisition-related  ques¬ 
tions  and  receive  formal  responses.  In  addition,  the  AAP  contains 
a  database  of  questions  and  answers  that  are  categorized  by  subject  area 
and  can  be  browsed  or  searched. 

Acquisition  Community  Connection  (ACC) 

The  ACC  <https://acc.dau.mil/>  is  the  collaborative  component  of 
the  AKMS  that  focuses  on  acquisition-related  topics  and  disci¬ 
plines  such  as  contracting,  logistics,  program  management,  and 
risk  management.  It  consists  of  Communities  of  Practice,  Special  Interest 
Areas,  and  collaborative  workspaces  that: 

•  connect  people  with  know-how  across  DoD  organizations  and  industry; 
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•  enable  members  to  interact  and  share  resources,  ideas,  and 
experiences  to  support  job  performance  and  avoid  duplication  of  effort; 
and 

•  identify  partnership  development  opportunities. 

Members  may  request  workspaces  in  ACC,  which  provide  a  way  for  physi¬ 
cally  dispersed  individuals  to  centrally  locate  and  share  documents  and 
references  as  well  as  manage  team  projects.  To  learn  more,  go  to  the  ACC 
<https://acc.dau.mil/>  and  take  the  online  virtual  tour. 

Defense  Acquisition  Guidebook  (DAG) 

The  DAG  <https://acc.dau.mil/dag>  provides  links  to  policy,  law, 
and  useful  content  housed  in  communities  of  practice.  It  allows 
users  to  navigate  through  the  DAG  via  a  document  index,  graphi¬ 
cal  interface  (Life  Cycle  System),  or  a  search  by  topic. 

ACQuipedia 

The  ACQuipedia  <https://acquipedia.dau.mil/>  is  your  acquisi¬ 
tion  encyclopedia  for  common  defense  acquisition  topics.  Each 
topic  is  identified  as  an  article;  each  article  contains  a  definition,  a 
brief  narrative  that  provides  context,  and  links  to  the  most  pertinent  policy, 
guidance,  tools,  practices,  and  training.  Articles  aggregate  the  most  relevant 
references  and  learning  assets  to  narrowly  focus  users  and  quickly  provide 
high-value  content. 


Integrated  Defense  AT&L  Life  Cycle  Management 
System  Chart 


(See  next  page  for  details) 
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This  chart  <https://acc.dau.mil/ifc>  is  a  pictorial  road  map  of  key 
—  activities  in  the  systems  acquisition  process.  Users  navigate 
through  a  graphical  model  of  the  three  major  acquisition  process 
areas:  Joint  Capabilities  Integration  and  Development  System  (JCIDS);  De¬ 
fense  Acquisition;  and  Planning,  Programming,  Budgeting,  and  Execution 
(PPB&E).  The  short  title  for  this  chart  is  Integrated  Life  Cycle  (ILC)  Chart. 


AT&L  ACQuire 

ACQuire  <https://acquire.dau.mil/>  is  a  search  tool  focused  on  the 
specific  needs  of  the  Defense  Acquisition  Workforce.  It  uses  the 
DAU  acquisition  taxonomy,  trusted  acquisition  sites,  and  selected 
AT&L  resources  to  enhance  searches  and  derive  better  results.  Searches 
can  be  conducted  by  individual  or  multiple  sites;  document  titles;  topic;  con¬ 
tent,  via  an  index  of  major  categories;  and  subcategories. 

Courseware  is  also  searchable  via  ACQuire.  Users  can  suggest  additional 
AT&L  sites  that  should  be  included  in  ACQuire  crawls. 


Best  Practices  Clearinghouse  (BPCh) 

The  BPCh  <https://bpch.dau.mil/Pages/default.aspx>  is  an  inno- 
vative  “clearinghouse”  approach  that  will  improve  all  DoD  acquisi¬ 
tion  processes  by  helping  AT&L  professionals  select  and  imple¬ 
ment  proven  practices  appropriate  to  the  individual  program  needs.  Initially, 
the  BPCh  will  focus  on  software  acquisition  and  systems  engineering. 

The  Clearinghouse  provides: 

•  an  authoritative  source  for  practices,  lessons  learned,  and  risks  to 
avoid; 

•  validated  practices  with  consistent,  verifiable  information; 

•  an  active  knowledge  base  to  help  with  practice  questions; 

•  an  intelligent  front-end  to  quickly  get  to  answers; 

•  useful  information  and  tools  to  help  find,  select,  and  implement 
practices  appropriate  to  specific  programs;  and 

•  living  knowledge  through  a  constantly  updated,  expanded,  and  refined 
database. 


Performance  Learning  Tools 

Performance  Learning  Tools  (PLTs)  link  learning  and  job  support  assets  to 
complicated  process  flow  to  help  users  create  plans  and  other  AT&L  prod¬ 
ucts  accurately  and  efficiently.  The  following  PLTs  have  been  developed: 

•  Pricing  Support  Tool  <http://pricingtool.dau.mil/> 

•  Performance-Based  Logistics  Toolkit  <https://acc.dau.mil/pbltoolkit> 
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DEFENSE  ACQUISITION  BOARD  TIMELINE  — 
MILESTONES  B,  C,  AND  FRPDR 


MILESTONE  DECISION  INFORMATION  — 
A  POSSIBLE  CONSTRUCT 


M  ^ 

^  WHY? 

•Threat 

•  Capability  ^ 

r?  1 

^  WHAT? 

•  Requirement 
•  Analysis  of  Alternatives 

73  ^ 

HOW? 

•  Acquisition  Strategy 

V  y 

^4  RISKS?  ^ 

•  Risk  Management  Plan 
•Test  &  Evaluation  Plan 
^  •Test  &  Evaluation  Results  ^ 

"5  COSTS?  ' 

•  CAIV  Objectives 
•  Life  Cycle  Cost  Estimate 
•  Independent  Cost  Estimate  ^ 

"6  MANAGEMENT?  ^ 

•  Program  Management  Office  Structure 
•  Integrated  ProductTeam 
•WIPT-OlPT  Structure 

"7  AGREEMENT?  ' 

•  Acquisition  Program  Baseline 
•  Acquisition  Decision  Memorandum 
•  Exit  Criteria 


LEGEND: 

CAIV— Cost  as  an  Independent  Variable 
DIPT— Overarching  Integrated  Product  Team 
WIPT— Working-Level  Integrated  Product  Team 


•  Have  I  presented  all  necessary  information?  •  Is  the  information  clear  and  accurate? 

•  Does  the  Information  flow  logically?  •  Is  it  concise,  executive-level  Information? 
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REQUIREMENTS  FOR  MILESTONE/DECISION  REVIEWS 

See  end  4,  DoDI  5000.02 


Requirement 

Milestone/Decision  Point 

MDD 

A 

B 

P-CDRA 

c 

FRP 

Acquisition  Decision  Memorandum® 

X 

X 

X 

X 

X 

X 

Acquisition  Program  Baseline® 

X 

X 

X 

Acquisition  Strategy®  (see  Page  5) 

X 

X 

X 

Acquisition  Information  Assurance  Strategy  (all  IT  inci  NSS) 

X 

X 

X 

X 

Affordability  Assessment 

X 

X 

Alternate  LF  T&E  Plan  (programs  with  waiver  from 
full-up  LFT&E)2 

X 

Analysis  of  Alternatives  (AoA)®  ® 

X 

X 

X 

X 

AoA  Study  Guidance 

X 

Benefit  Analysis  &  Determination'  ®  (bundled  acquisitions) 

X 

Beyond  LRIP  Report®  (inci  MDAPs  that  are  also  MAIS) 

X 

Capability  Development  Document  (CDD)® 

X 

Capability  Production  Document  (CPD) 

X 

Clinger-Cohen  Act  (CCA)  Compliance®  ® 

X 

X 

X 

X 

Competition  Analysis'  ®  (depot-level  maintenance  rule) 

X 

CIO  Confirmation  of  CCA  Compliance 
(DoD  CIO  for  MAIS  and  MDAP) 

X 

X 

X 

X 

Component  Cost  Estimate®  "  (MAIS;  optional  MDAP) 

X 

X 

X 

Consideration  of  Technology  Issues  (MDAP  &  MAIS) 

X 

X 

X 

Cooperative  Opportunities' 

X 

X 

X 

Core  Logistics/Source  of  Repair  Analysis'  ® 

X 

X 

Corrosion  Prevention  Control  Plan' 

X 

X 

Cost  Analysis  Requirements  Description®  "  (MDAP  &  MAIS) 

X 

X 

X 

Data  Management  Strategy'  (MDAP,  MAIS  &  ACAT II) 

X 

X 

X 

X 

Economic  Analysis  (MAIS)®  (may  be  combined  w/AoA  at  MSA) 

X 

X 

X 

Exit  Criteria® 

X 

X 

X 

X 

Industrial  Base  Capabilities'  (MDAPs  only) 

X 

X 

X 

Independent  Cost  Estimate  (ICE)®  '"  '®  (MDAPs  only) 

X 

X 

X 

X 

Independent  Technology  Readiness  Assessment " 

X 

X 

Information  Support  Plan'  ® 

X 

X 

X 

Initial  Capabilities  Document  (ICD)*® 

X 

X 

X 

X 

Initial  Operational  Test  &  Evaluation  Completed  (ACAT  I  &  II) 

X 

Item  Unique  Identification  (lUID)  Plan  (Part  of  SEP) 

X 

X 

X 

Joint  Interoperability  Test  Certification  (IT  &  NSS) 

X 

Life-Cycle  Signature  Support  Plan® 

X 

X 

X 

Life-Cycle  Sustainment  Plan' 

X 

X 

X 

LF  T&E  Waiver"  (covered  systems)  (n/a  MAIS) 

X 

LF  T&E  Reoort"  (covered  systems)  (n/a  MAIS) 

X 
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Requirement 

Milestone/Decision  Point 

MDD 

A 

B 

P-CDRA 

c 

FRP 

LRIP  Quantities  MDAP  &  ACAT  II  (n/a  AIS) 

X 

Manpower  Estimate  (MDAPS  only)' 

X 

X 

X 

Market  Research 

X 

X 

Milestone  Decision  Authority  (MDA)  Certification  (MDAPs  only)“ 

X 

X 

X 

MDA  assessment  of  chem,  bio,  rad,  and  nuc  survivability 

X 

X 

Military  Equipment  Validation' 

X 

X 

Net-Centric  Data  Strategy'  ’^ 

X 

X 

X 

Operational  Test  Agency  Report  of  OT&E  Results 

X 

X 

X 

Preliminary  Design  Review  (PDR)  Report’' 

X 

Post-Critical  Design  Review  (CDR)  Report 

X 

Post  Implementation  Review 

X 

Program  Protection  Plan  (PPP)’ 

X 

X 

X 

Prog  Environment,  Safety  &  Occup  Flealth  Evaluation  (PESFIE)' 

X 

X 

X 

Replaced  System  Sustainment  Plan'  (MDAPs  only) 

X 

Selected  Acquisition  Report  (SAR)“’'“ 

Spectrum  Supportability  Determination® 

X 

X 

System  Threat  Assessment  (STA)  (ACAT  11)'  ’“ 

X 

X 

System  Threat  Assessment  Report  (STAR)  (ACAT  1)'  ' 

X 

X 

Systems  Engineering  Plan  (SEP) 

X 

X 

X 

Technology  Development  Strategy  (TDS) 

X 

X 

X 

Technology  Readiness  Assessment  (TRA)' 

X 

X 

Test  &  Evaluation  Master  Plan  (TEMP) 

X 

X 

X 

Test  &  Evaluation  Strategy  (TES) 

X 

'  Part  of  TDS  or  Acquisition  Strategy 
^OSD  T&E  Oversight  Programs 
=MDAP:A,B,C;  MAIS:A,  B,  FRP 
“Milestone  C  if  Program  Initiation 
'^Program  Initiation  for  Ships 

'Validated  by  DIAfor  ACAT  ID;  AIS  use  DIA  validated  capstone  info/ops  Threat  Assessment  Decision 

'Milestone  C  if  equivalent  to  FRP 

“Milestone  C  if  no  Milestone  B 

“  MAIS  whenever  an  economic  analysis  is  required 

’“May  be  CAIG  Assessment  at  Milestone  A 

”ACAT  ID  only  if  required  by  DDR&E 

’“Summarized  in  TDS;  details  in  ISP 

’“SAR  at  program  initiation;  annually  thereafter 

’“Validated  by  Component;  AIS  use  DIA  validated  capstone  info/ops  Threat  Assessment  Decision 

’“  MDA  may  request  cost  assessment  at  Milestone  A 

’“  If  PDR  is  after  Milestone  B,  MDA  will  conduct  a  post-PDR  assessment  review 
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JOINT  CAPABILITIES  TECHNOLOGY 
DEMONSTRATION  (JCTD) 


JCTD  Identification,  Selection,  and  Review 


Annual  Review  and  Approval  for  New  Fiscal  Year  Starts 


Legend; 

CDB  —  Candidate  Decision  Board 
CNB  —  Candidate  Nomination  Board 
PRB  —  Project  Review  Board 


JCTD  EXECUTION  APPROACH 

Annual  call  for  proposals;  proposals  submitted  at  any  time 

•  Candidate  projects  identified  by  Combatant  Commands,  Services  and 
Agencies,  Interagency  or  Coalition  Partners,  and  industry 

•  Joint/Interagency  Government-led  teams  build  proposals 

•  Technical  manager  from  Service  or  Agency 

•  Operational  manager  from  Combatant  Command  chain-of-command 

•  Transition  manager  from  Acquisition  Community 

•  USD(AT&L)  approval  of  recommended  new  starts 

•  Congressional  Notification 

•  DDR&E/Rapid  Fielding  Directorate  oversight  during  project  execution 

•  Operational  demonstration  and  rigorous  assessment 

•  Continuous  transition  of  capabilities  demonstrating  required 
capabilities 

Beginning  in  FY10: 

•  Quarterly  Candidate  Decision  Boards  to  recommend  new  starts, 
chaired  by  DDR&E,  co-chaired  by  Joint  Staff  Deputy  J8 

•  Quarterly  Review  Boards  to  review  ongoing  projects 

•  More  short  projects  (1  year  or  less);  fewer  long-term  projects  (more 
than  1  year) 

•  All  projects  structured  for  deliverables  in  first  year,  with  annual  review 

•  JROC  validation  following  first  year  of  execution 
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LEGEND: 

ACAT— Acquisition  Category  DT/OT— Deveiopmentai/Operationai  Testing  MUA— Miiitary  Utiiity  Assessment 

CDD— Capabiiity  Deveiopment  Document  FAR— Federai  Acquisition  Reguiation 

CPD— Capabiiity  Production  Document  FYDP— Future  Years  Defense  Program 

DDR&E— Director,  Defense  Research  &  Engineering  ICD— Initiai  Capabiiities  Document 


PROGRAM  STRUCTURE  (Example — No  Streamlining) 
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DOD  INTERNATIONAL  ARMAMENTS 
COOPERATION  POLICY 

“PMs  shall  pursue  international  armaments  cooperation  to  the 
maximum  extent  feasible,  consistent  with  sound  business  practice 
and  with  the  overall  political,  economic,  technological,  and  national 
security  goals  of  the  United  States.  International  agreements  for 
international  armaments  cooperation  programs  shall  complete  the 
interagency  consultation  and  Congressional  notification  requirements 
contained  in  10  U.S.C.  2350a,  Section  2751  of  the  Arms  Export 
Control  Act,  and  10  U.S.C.  2531.” 


-DoDD  5000.01  (Para  El. 1.1) 


THE  SCOPE  OF  DEFENSE  COOPERATION 


RDT&E 

Production  and 
Procurement 

Follow-on  Support 

Information 

Exchanges 

Foreign  Military  Sales 

Cooperative  Logistics 
Supply  Support 

Engineer  and 

Scientist  Exchanges 

Direct  Commercial  Sales 
Exchanges 

Mutual  Support 

Cooperative  R&D 

Cooperative  Production 
(Joint  Funds) 

Logistics  Support 

Comparative  or 

Joint  Testing 

Standardization 

Coproduction/Licensing 
(Foreign  Funds) 

Reciprocal  Procurement 

Host  Nation  Support 
Defense  Industrial 
Base 

t  t 


The  Program  Manager’s  Focus 
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DEFENSE  SALES  vs.  COOPERATIVE  ACQUISITION 
They  are  Different 


Defense  Sales 

-  Any  Nation 

-  U.S.  Contracts  (FMS) 

-  U.S.  Manages  (FMS) 

-  Production  and  Support 

-  Dept,  of  State  or  Dept, 
of  Commerce  +  DoD 
(USD(Policy)) 

-  Foreign  Initiated 

-  Foreign  Funds  (or  U.S. 
Credit/Grants) 


Cooperative  Acquisition 

-  Allied  or  Friendly 

-  U.S.,  Ally  or  NATO 

-  Jointly  Managed 

-  All  Acquisition 

-  DoD  (USD(AT&L))  + 

Dept,  of  State  and 
Dept,  of  Commerce 

-  U.S.  and/or  Foreign  Initiated 

-  Foreign  +  U.S.  Funds 


INTERNATIONAL  ACTIVITIES  ASSOCIATED  WITH 
DEFENSE  ACQUISITION  PHASES 


Cooperative  Productioa 
Coproduction  - 

■  Licensed  Production 
Production  Sharing 
Foreign  Military  Sales— 


Production 

and 

Deployment, 

Sustainment 


Cooperative  Development - ► 

International  Testing - ► 


System  Capability  and 
Manufacturing  Process 
Demo  of  EMD  Phase 


NATO  Forums- 
DEAs/IEPs  — 
Staff  Talks  — 


i 


S&E  Exchangesj - 

Technology  Opportunities 
and  User  Capability  Needs 


Materiel  Solution  Analysis 
and 

Technology  Development 


LEGEND: 

DEA— Data  Exchange  Agreement 
EMD— Engineering  &  Manufacturing  Development 
lEP— Information  Exchange  Project 
S&E— Science  and  Engineering 
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RESOURCE  ALLOCATION  PROCESS— OVERLAP 


DAU  PROGRAM  MANAGERS  TOOL  KIT 


?  ?  ^  ^  ^ 
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PPBE— PLANNING  PHASE 


National  Strategy  Documents 
Provide  Input  to  PPBE 


FEB 


MAR 


APR 


MAY 


JUN 


President 

National  Security  Council 


— I  Front  End  Assessments  |-» 


—  Joint  Strategy  Review(Joint  Staff,  Combatant  Commands,  Military  Departments) — > 


LEGEND: 

CPR-Chairman’s  Program  Recommendation  DPPG-Defense  Planning  &  Programming  Guidance 
CJCS— Chairman,  Joint  Chiefs  of  Staff  QDR— Quadrennial  Defense  Review 


PPBE— PROGRAM/BUDGET  REVIEW 

JUL  AUG  OCT  NOV  DEC  FEB 

I  Front  End  Assessments  \ 


OSD  Budget  Execution  &  Program  Performance  Review 


LEGEND: 

BES-Budget  Estimate  Submission  FYDP-Future  Years  Defense  Program 

CPA-Chairman’s  Program  Assessment  POM-Program  Objectives  Memorandum 

DAWG— Deputy’s  Advisory  Working  Group 
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RESOURCE  ALLOCATION  PROCESS 


DAU  PROGRAM  MANAGERS  TOOL  KIT 
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CONGRESSIONAL  ENACTMENT  TIMETABLE 


DAU  PROGRAM  MANAGERS  TOOL  KIT 
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PROCUREMENT  APPROPRIATIONS 

(Account  Numbers  and  Budget  Activities) 


Appropriation 

Budget  Activity 

Aom^i  -J 

Aircraft 

-  2031  1  Aircraft 

2  Modification  of  Aircraft 

3  Spares  and  Repair  Parts 

4  Support  Equipment  and  Facilities 

Missile 

-  2032  1  Not  Used 

2  Other  Missiles 

3  Modification  of  Missiles 

4  Spares  and  Repair  Parts 

5  Support  Equipment  and  Facilities 

Weapons  and  Tracked 

-  2033  1  Tracked  Combat  Vehicles 

Combat  Vehicles 

2  Weapons  and  Other  Combat  Vehicles 

3  Spares  and  Repair  Parts 

Ammo 

-  2034  1  Ammo 

2  Ammo  Production  Base  Support 

Other 

-  2035  1  Tactical  and  Support  Vehicle 

2  Communications  and  Electronics 

3  Other  Support  Equipment 

4  Initial  Spares 

\Namr(i7-t 

Aircraft 

-  1506  1  Combat  Aircraft 

2  Airlift  Aircraft 

3  Trainer  Aircraft 

4  Other  Aircraft 

5  Modification  of  Aircraft 

6  Aircraft  Spares  and  Repair  Parts 

7  Aircraft  Support  Equipment  and  Facilities 

Weapons 

-  1507  1  Ballistic  Missiles 

2  Other  Missiles 

3  Torpedoes  and  Related  Equipment 

4  Other  Weapons 

5  Not  Used 

6  Spares  and  Repair  Parts 

Ammo,  Navy  and 

-  1508  1  Ammo,  Navy 

Marine  Corps 

2  Ammo,  Marine  Corps 

Shipbuilding 

- 161 1  1  Not  Used 

and  Conversion 

2  Other  Warships 

3  Amphibious  Ships 

4  Not  Used 

5  Auxiliaries,  Craft,  and  Prior- Year  Program  Costs 

Other 

- 1810  1  Ships  Support  Costs 

2  Communications  and  Electronics  Equipment 

3  Aviation  Support  Equipment 

4  Ordnance  Support  Equipment 

5  Civil  Engineering  Support  Equipment 

6  Supply  Support  Equipment 
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DAU  PROGRAM  MANAGERS  TOOL  KIT 


PROCUREMENT  APPROPRIATIONS 

(Account  Numbers  and  Budget  Activities)  (Continued) 


Appropriation  Budget  Activity 


Other  (continued)  -1810  7  Personnel  and  Command  Support  Equipment 
8  Spares  and  Repair  Parts 

Marine  CBWB{17  -J 


Procurement 

-1109 

1  Not  Used 

2  Weapons  and  Combat  Vehicles 

3  Guided  Missiles  and  Equipment 

4  Communications  and  Electronics  Equipment 

5  Support  Vehicles 

6  Engineering  and  Other  Equipment 

7  Spares  and  Repair  Parts 

1  Air  Force  {57  -J 

Aircraft 

-3010 

1  Combat  Aircraft 

2  Airlift  Aircraft 

3  Trainer  Aircraft 

4  Other  Aircraft 

5  Modification  of  In-Service  Aircraft 

6  Aircraft  Spares  and  Repair  Parts 

7  Aircraft  Support  Equipment  and  Facilities 

Missile 

-3020 

1  Ballistic  Missiles 

2  Other  Missiles 

3  Modification  of  In-Service  Missiles 

4  Spares  and  Repair  Parts 

5  Other  Support 

Ammo 

-3011 

1  Ammo 

2  Weapons 

Other 

-3080 

1  Not  Used 

2  Vehicular  Equipment 

3  Electronics  and  Telecommunications  Equipment 

4  Other  Base  Maintenance  and  Support  Equipment 

5  Spares  and  Repair  Parts 

1  Defense  {97  -} 

Defense-wide 


National  Guard 
and  Reserve 
Equipment 
Defense  Production 
Activity 
Purchase 
Chemical  Agents 
and  Munitions 
Destruction 


-  0300  1  Major  Equipment 

2  Special  Operations  Command 

3  Chemical/Biological  Defense 

-  0350  1  Reserve  Equipment 

2  National  Guard  Equipment 

-  0360  1  Defense  Production  Activity  Purchases 


-  0390  1  Chemical  Agents  and  Munitions  Destruction — O&M 

2  Chemical  Agents  and  Munitions  Destruction — RDT&E 

3  Chemical  Agents  and  Munitions  Destruction — 
Procurement 


Rapid  Acquisition  Fund  -2095  1  Rapid  Acquisition  Fund 
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RESEARCH,  DEVELOPMENT, TEST,  AND  EVALUATION 
(RDT&E)  APPROPRIATIONS 
(Account  Numbers) 


Appropriation 

Account 

Number 

RDT&E,  Army 

21  -2040 

RDT&E,  Navy 

17-1319 

RDT&E,  Air  Force 

57  -  3600 

RDT&E,  Defense-wide 

97  -  0400 

Development  T&E,  Defense 

97  -  0450 

Operational  T&E,  Defense 

97  -  0460 

RDT&E  APPROPRIATIONS 
(Relationship  Between  MFP  6  R&D  Categories  and 
RDT&E  Appropriations  Budget  Activities) 


MFP  6 

RDT&E 

R&D 

Budget 

RDT&E 

Cateaorv 

Activity 

Budoet  Activity  Title 

6.1 

BA  1 

Basic  Research 

6.2 

BA  2 

Applied  Research 

6.3 

BA  3 

Advanced  Technology  Development 

6.4 

BA  4 

Advanced  Component  Development  and 
Prototypes 

6.5 

BA  5 

System  Development  and  Demonstration 

6.6 

BA  6 

RDT&E  Management  Support 

... 

BA  7 

Operational  System  Development 

LEGEND: 

BA— Budget  Activity 

R&D— Research  and  Development 

MFP— Major  Force  Program 

T&E— Test  and  Evaluation 

•NOTE:  Although  similar,  titles  of  the  Major  Force  Program  (MFP)  6  categories  (which  are  not  shown  above) 
are  not  exactly  the  same  as  titles  of  the  RDT&E  Appropriation  Budget  Activities.  In  addition,  the  “Operational 
System  Deveiopmenf  Budget  Activity  for  RDT&E  BA  7  is  not  considered  MFP  6.  While  correctly  funded 
with  RDT&E  dollars,  these  efforts  do  not  fall  under  a  MFP  6  Category;  rather,  for  MFP  purposes,  the  efforts 
are  considered  part  of  the  MFP  that  the  fielded  operational  system  falls  within.  Congress  calls  BA  4, 
Demonstration  and  Validation,  and  calls  BA  5,  Engineering  and  Manufacturing  Development. 
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SAMPLE  NAVY  APPROPRIATIONS  AND 
BUDGET  ACTIVITIES 


MFP6 

R&D 


RDT&E  Budget  Activity  (BA) 


Beiow  Threshold 
Reprogramming  Ruies 


Years  Avaiiable 
for  Obiigation 


Funding 


Category 

Number  and  Title 

Max  In* 

Max  Out* 

Purposes 

Policy 

(At  Prog.  Element  Level) 

6.1 

BA1 

Basic  Research 

Lesser  of 

Leaser  of 

Incremental 

6.2 

BA  2 

Applied  Research 

+$10M  or 

-$10M  or 

6.3 

BA  3 

Advanced  Tech.  Development 

+20%  of 

-20%  of 

6.4 

BA  4 

Adv.  Comp.  Dev.  and  Prototypes 

amount 

amount 

6.5 

BAS 

System  Devel.  and  Demo. 

appropriated 

appropriated 

6.6 

BA  6 

RDT&E  Management  Support 
(T&E  Ranges)  (Civilian  Salaries) 

1 

I 

BA  7 

Operational  Systems  Devel. 
(Post-Production) 

1 

1 

1 

Procurement 

Budget  Budget  Activity 

Activity  Description 


Beiow  Threshoid 
Reprogramming  Rules 
Max  In  Max  Out 

(At  Line  Item  Level) 


Years  Available 
for  Obligation 
Purposes 


Funding 

Policy 


PROCUREMENT  (Proc.) 

SCN-1  Not  Used  Lesser  of 

SCN-2  Ship  Conversion— Other  Warships  +$20M  or 

SCN-3  Ship  Conversion— Amphibious  Ships  +20%  of 

SCN-4  Not  Used  amount 

SCN-5  Ship  Conversion— Auxiliaries,  Craft,  and  appropriated 

Prior- Year  Program  Costs 

WPN-1  Weapons  Proc— Ballistic  Missiles 
WPN-2  Weapons  Proc— Other  Missiles 
WPN-3  Weapons  Proc— Torpedos  and  Equipment 
WPN-4  Weapons  Proc— Other  Weapons 
WPN-5  Not  Used 

WPN-6  Weapons  Proc— Spares  and  Repair  Parts 

OPN-1  Other  Proc— Ship  Support  Equipment  (SE) 

OPN-2  Other  Proc— Comm./Electronics  Equip. 

OPN-3  Other  Proc— Aviation  SE 

OPN-4  Other  Proc— Ordnance  SE 

OPN-5  Other  Proc— Civil  Engineering  SE 

OPN-6  Other  Proc— Supply  SE 

OPN-7  Other  Proc— Pers.  and  Command  SE 

OPN-8  Other  Proc— Spares  and  Repair  Parts 

APN-1  Aircraft  Proc— Combat  Aircraft 

APN-2  Aircraft  Proc— Airlift  Aircraft 

APN-3  Aircraft  Proc— Trainer  Aircraft 

APN-4  Aircraft  Proc— Other  Aircraft 

APN-5  Aircraft  Proc— Modifications  of  Aircraft 

APN-6  Aircraft  Proc— Aircraft  Spares  and  Repair  Parts 

APN-7  Aircraft  Proc— Aircraft  SE  and  Facilities 


Lesser  of 
-$20M  or 
-20%  of 
amount 
appropriated 


Full 


t 

3 


Below  Threshold 
Reprogramming  Rules 


Years  Available 
for  Obligation 


Funding 


Other  Appropriations  /Titles 

Max  In 

Max  Out  Purposes 

Policy 

O&M,  N  Operations  and  Maintenance 

$15M 

No  Congressional  1 

Restriction 

Annual 

MILPER,  N  Military  Personnel 

$10M 

No  Congressional  1 

Restriction 

Annual 

MILCON,  N  Military  Construction 

Lesser  of 
+$2.0M  or +25% 
Appropriated 

No  Congressional  5 

Restriction 

Full 

'Below  Threshold  Reprogramming  (BTR)  amount  limits  are  cumulative  over  entire  period  of  time  the 
specific  fiscal  year  appropriation  is  available  for  obligation  purposes  (i.e.,  1 , 2,  3,  or  5  years). 
"Reference  Source:  DoD  FMR,  Volume  3,  Chapter  6,  a/o  September  201 0. 
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APPROPRIATION  LIFE 


YEARS 


CO 

CM 

4 

5 

6 

7 

8 

9 

10 

11 

O&M 

TOW 

TOTO 

WTO 

1 

RDT&E 

PROCUREMENT 

TO 

TO 

TO 

SHIPS 

Vio:o!«I4 

MILCON 

TOTO 

TOTO 

TOTO 

TOTO 

1  1 

MILPERS 

mm 

TO 

TO 

TO 

r 

Current  Period;  Available  for  new  obligations,  obligation  adjustments,  expenditures,  and  outlays 
Expired  Period;  Available  for  obligation  adjustments,  expenditures,  and  outlays 
Y/////\  Cancelled:  Unavailable  for  obligations,  obligation  adjustments,  expenditures,  and  outlays 


BELOW  THRESHOLD  REPROGRAMMING  ACTIONS 
Amounts  are  Cumulative  Over  Entire  Period  of 
Obligation  Availability 


APPN 

MAX  INTO 

MAX  OUT 

LEVEL  OF  CONTROL 

OBLIGATION 

AVAILABLE 

RDT&E 

Lesser  of 
+$10M  or 
+20% 

Lesser  of 
-$10M  or 
■20% 

Program  Element 

2  Years 

PROC 

Lesser  of 
+$20M  or 
+20% 

Lesser  of 
-$20M  or 
■20% 

Line  Item 

3  Years 
(Shipbuilding 
and  Conversion, 
Navy:  SYears) 

O&M 

+$15M 

None,  Unless 
Otherwise 
Specified 

Budget  Activity  (BA) 

Some  BA  1  Subjectivity 
Limitations  on  Decreases 
(Operating  Forces) 

lYear 

MILPERS 

+$10M 

No  Specific 
Congressional 
Restriction 

Budget  Activity 

lYear 

MILCON 

Lesser  of 
+$2M 
+25% 

No  Specific 
Congressional 
Restriction 

Project 

SYears 

Reference  Sources:  (1)  USD  (C)  Memo;  SUBJECT:  FY  2006  Below  Threshold  Reprogramming  Authority  Policy, 
10  Feb  2006,  (2)  USD  (C)  Memo,  SUBJECT:  DD  1414,  Base  for  Reprogramming  Actions,  5  Jan  2010 
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LIFE  CYCLE  COST  COMPOSITION 


Life  Cycle  Cost 


Program /^quisjtion_Cost 


MILCON 

Facilities 


Procurement  Cost 
Weapon  System  Cost 
Flyaway  Cost 


PROCUREMENT 
Prime  Mission 
Equipment  (PME) 


PROCUREMENT 

1 

PROCUREMENT 

Support  Items 

1 

Initial  Spares 

Development  Cost 


RDT&E 

Development  Costs  of 
PME  and  Support  Items 
Systems  Engineering 
Program  Management 
Test  and  Evaluation 


J 


Operations  and 

Support 

•  0&M,MILPERS 
(or  others  as 
appropriate) 


Disposal 
•  O&M  (or  others 
as  appropriate) 


PRODUCT  IMPROVEMENTS 

Funding  Decision  Tree 


IF. 


Does  Proposed 
Modification  (Mod) 
Increase  System 
Performance? 


YES 


THEN  . . . 

Fund  Development 
and  Testing  with  . . . 
(To  Include  the  Mod 
Kits  used  for  Testing) 


AND . . . 

Fund  Purchase  of  the  Mod  Kits  and 
Installation  of  those  Mod  Kits  on  the 
Fielded  System  with  . . . 


Is  DT  or  lOT&E 
Required? 


Is  System 
Currently  In 
Production? 


rNO 


O&M  I 
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COST  ESTIMATING 

Estimate  Methods  Comments 


Analogy 


Parametric 


Engineering  or 
“Bottoms-Up” 


Comparison  to  one  similar  existing  system; 
based  on  judgments.  Little  or  no  data  available; 
relatively  quick,  easy,  flexible.  Used  in  early  phases 
(e.g.,  Material  Solution  Analysis  and  Tech.  Dev.) 

Comparison  to  many  similar  existing  systems; 
based  on  statistical  analysis.  Determine 
primary  cost  drivers  and  establish  Cost 
Estimating  Relationships  (CERs).  Used  in  early 
to  mid-phases  (e.g.,  Material  Solution  Analysis, 
Tech.  Dev.,  and  Engr.  and  Manufacturing  Dev.) 

Summation  of  “all”  individual  items  in  the  system. 
Uses  Work  Breakdown  Structure  (WBS) 
for  estimating  purposes.  Used  in  mid-phases 
(e.g.,  Engineering  and  Manufacturing  Development) 


Extrapolation  Comparison  to  historical  cost  of  same  system. 

Based  on  extrapolation  from  actuals.  Uses 
Learning  Curve  Theory.  Used  in  late  phases 
(e.g..  Production  and  Operations/Support) 

Guidelines 

1 .  Make  sure  cost  data  are  relevant  and  homogeneous.  Caution:  Watch  out  for 
historical  data  in  times  of  change.  Prior  actuals  may  include  uncompensated 
overtime  or  were  priced  as  a  “buy-in.” 

2.  Focus  on  cost  drivers. 

3.  Test  sensitivities  and  data  relationships. 


COST  ESTIMATING  RELATIONSHIPS  (CER)— 
PARAMETRIC 
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COST  ESTIMATING  REQUIREMENTS 

ACAT  1C  and  ID  (MDAP)  ACAT  lAM  and  lAC  (MAIS) 

Program  initiation  &  aii  subsequent  Program  initiation  &  aii  subsequent 

miiestones,  inciuding  FRP  DR  miiestones 


CARD  Program  initiation  &  aii  subsequent  Program  initiation  &  whenever 
miiestones  inciuding  FRP  DR  Economic  Analysis  is  required 

•  Draft:  1 80  days  prior  to  OiPT  •  Draft:  1 80  days  prior  to  OiPT 

•  Finai:  45  days  prior  to  OiPT  •  Finai:  45  days  prior  to  OiPT 


CCE  A  and  aii  subsequent  miiestones  MS  A  and  whenever  an  Economic 
inciuding  FRP  DR  Analysis  inciuding  is  required 


ICE  Required  by  iaw  for  aii  MDAP  programs  * 

•  Prepared  by  OSD  CAPE  for  ACAT  iD,  and  ACAT  iCat  discretion  of  USD  (AT&L) 

•  Prepared  by  component  cost  agency  (AFCAA,DASA-CE,  NCCA)  for  ACAT  iC  (if  no 
CAPE  estimafe) 

•  in  advance  of  any  certificafion  under  Titie  10,  U.S.C.,  Section  2366a  (MSA)  and 
Section  2366b  (MS  B) 

•  in  advance  of  any  decision  fo  enter  iow-rate  initiai  production  (LRiP)  (MS  C)  or 
fuii-rate  production  (FRP  DR) 

•  in  advance  of  any  certificafion  of  MDAPs  that  experience  criticai  cost  growth  (Titie 
10,  U.S.C.,  Sec  2433a) 

•  in  advance  of  any  report  of  Criticai  Program  Changes  for  MAiS  (Titie  10,  U.S.C., 
Sec  2445c(f)) 

*ICE  statutory  requirement  (Title  10,  US  Code,  Sec  2434)  and  PI.  111-23,  May  22, 2009 
Source:  DoDI5000.02,  December  2008  and  Weapon  Systems  Acquisition  Reform  Act  of  2009 


ACAT  II  &  ACAT  III:  POE  (and,  at  MDA  discretion,  an 
independent  cost  estimate  prepared  by  the  component  cost 
agency)  at  program  initiation  and  all  subsequent  milestones 


LEGEND: 

AFCAA— Air  Force  Cost  Analysis  Agency 
CAE— Component  Acquisition  Executive 
CAPE— Cost  Assessment  &  Program  Evaluation 
CARD— Cost  Analysis  Requirements  Description 
CCE— Component  Cost  Estimate 
DASA-CE— DepAsst  Sec  of  Army  (Cost  & 
Economics) 

FRP  DR— Full  Rate  Production  Decision  Review 
ICE— Independent  Cost  Estimate 


MAIS— Major  Automated  Information  System 
MDA— Milestone  Decision  Authority 
MDAP— Major  Defense  Acquisition  Program 
NCCA— Naval  Center  for  Cost  Analysis 
OIPT— Overarching  Integrated  Product  Team 
POE— Program  Office  Estimate 
USD(AT&L)— Under  Secretary  of  Defense 
(Acquisition,  Technology,  &  Logistics) 
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EARNED  VALUE  MANAGEMENT  “GOLD  CARD” 


VARIANCES  Favorable  is  Positive,  Unfavorabie  is  Negative 


Cost  Variance 

CV 

=  BCWP  - 

ACWP 

CV%  =  (CV/BCWP)*100 

Schedule  Variance 

SV 

=  BCWP  - 

BCWS 

SV%  =  (SV/ BCWS) ‘100 

Variance  at  Completion 

VAC 

=  BAC  - 

EAC 

OVERALL  STATUS 

%Seheduie  =  (BCWS,,„/BAC)  *  100 
%Compiete  =  (BCWP,,„/BAC)  *  100 
%  Spent  =  (ACWP,,„/BAC)*100 


DoDTRIPWIRE  METRICS  (TW) 

Cost  Efficiency  CPi  =  BCWP/ACWP  Favorabie  is  >  1 .0,  Unfavorabie  is  <  1 .0 

*™*Scheduie  Efficiency  SPi=BCWP/BCWS  Favorabie  is  >1.0,  Unfavorabie  is  <  1.0 

BASELINE  EXECUTION  INDEX  (BEI)  (Schedule  Metric) 

BEI  =  Tasks  with  Actual  Finish  Date  /  (#  of  Baseline  Tasks  Scheduled  to  Finish  Prior  to  Status 
Date  +  Tasks  Missing  Baseline  Start  or  Finish  Date) 


(TW) 


CRITICAL  PATH  LENGTH  INDEX  (CPU)  (Schedule  Metric) 

CPLI  -  (CP  Length  contract  End)  Float  Baseline  F|„js|,) 


)  /  CP  Length 


HIT/MISS  (Month’s  Task  Completed  ON  or  AHEAD/  Month’s  Tasks  Scheduled  to  Complete) 

TO  COMPLETE  PERFORMANCE  INDEX  (TCPI)*® 

TCPIbac  =  Work  Remaining  /  Cost  Remaining  =  (BAC  -  BCWP^^J  /  (EAC 

ESTIMATE  AT  COMPLETION  *  (EAC)  (Actuals  to  Date  +  [(Remaining  Work)/(Performance  Factor)] 
EAC,„  =  ACWP,,„  +  [(BAC  -  BCWP,  J/CPI,,,  ]  =  BAC/CPI,,, 

EACee^pos.  =  ACWP,,„  +[(BAC-BCWP,J/(CPI,,„*  SPI,J 

•To  determine  a  contract  level  TCPI  or  EAC,  you  may  replace  BAC  with  TAB. 

STo  determine  the  TCPI  substitute  BAC  or  LRE  for  EAC. 

dAO  or  Lnt 

For  more  information,  go  to  the  EVM  Home  Page  at  https://acc.dau.mil/evm,  e-mail  EVM  at 
EVM.dau@dau.mil,  or  call  703-805-5259  (DSN  655). 
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EARNED  VALUE  MANAGEMENT  “GOLD  CARD” 

(Continued) 

I  Contract  Price~~| 


I  Work  Packages 

TERMINOLOGY 

ACWP  Actual  Cost  of  Work  Performed 


=1 - , 

Planning  Packages  | 


AUW  Authorized  Unpriced  Work 
BAG  Budget  At  Completion 
BCWP  Budgeted  Cost  for  Work  Pert, 


CA  Control  Account 


Cost  actually  incurred  in  accomplishing  work  performed  = 

ACTUAL  COST 

Work  contractually  approved,  but  not  yet  negotiated /definitized 
Total  budget  for  total  contract  thru  any  given  level 
Value  of  completed  work  in  terms  of  the  work’s  assigned  budget  = 
EARNED  VALUE 

BCWS  Budgeted  Cost  for  Work  Sched.  Time-phase  Budget  Plan  for  work  currently  scheduled  = 

PLANNED  VALUE 

Lowest  CWBS  element  assigned  to  a  single  focal  point  to  plan  and 
control  scope/schedule/budget 
SumofNCC  and  AUW 

Estimate  of  total  Cost  for  total  contract  thru  any  given  level; 
may  be  generated  by  Ktr,  PMO,  DCMA,  etc.  =  EAC„„,p„o;ocma 
Ktr’sEACorEAC^P 

Budget  withheld  by  Ktr  PM  for  unknowns/risk  management 
Contract  Price  minus  profit /fee(s) 

Sum  of  CBB  and  recognized  overrun 
NCC  Plus  Profit  or  Fee(s) 

Contract  time-phased  budget  plan 
Far-term  CA  activities  not  yet  defined  into  WPs 
Far-term  activities  not  yet  defined  into  CAs 
Sum  of  all  budgets  for  work  on  confract  =  NCC,  CBB,  or  0TB 
To  Complete  Performance  Index  Efficiency  needed  from  ‘time  now’  to  achieve  an  BAC,  EAC,  or  LRE 
Undistributed  Budget  Broadly  defined  activities  not  yet  distributed  to  CAs 

Work  Package  Near-term,  detail-planned  activities  within  a  CA 


CBB 

EAC 

LRE 

MR 

NCC 

0TB 

PAC 

PMB 

PP 

SLPP 

TAB 

TCPI 

UB 

WP 


Contract  Budget  Base 
Estimate  At  Completion 

Latest  Revised  Estimate 
Management  Reserve 
Negotiated  Contract  Cost 
Over  Target  Baseline 
Price  At  Completion 
Pert.  Measurement  Baseline 
Planning  Package 
Summary  Level  Plan.  Package 
Total  Allocated  Budget 


EVM  POLICY:  DoDI  5000.02,  Table  E4.T5.  EVMS  in  accordance  with  ANSI/EIA-748  is  required  for  cost  or 
incentive  contracts,  subcontracts,  intra-government  work  agreements,  and  other  agreements  valued 
>  $20M  (Then-Yr  $).  EVMS  contracts  >  $50M  (TY  $)  require  that  the  EVM  system  be  formally  validated 
by  the  cognizant  contracting  officer.  Additional  guidance  is  in  the  Defense  Acquisition  Guidebook  and  the 
Earned  Vaiue  Management  Impiementation  Guide  (EVMIG).  EVMS  is  discouraged  on  Firm-Fixed  Price  & 
Tme  &  Material  Contracts,  &  LOE  activities  regardless  of  cost. 

EVM  CONTRACTING  REQUIREMENTS: 

FAR  EVM  Clauses  Non-DoD  —  52.234-2  for  Solicitation  (Pre-Award  ibr)  or  52.234-3  (Post  Award  ibr) 

—  52.234-4  Solicitation  &  Contract 

DoD  Use  DEAR  Clauses  { >  $20M)  -252.234-7001  “Notice  of  EVMS”  for  solicitations 

—  252.234-7002  “EVMS”  for  solicitations  &  contracts 

Contract  Performance  Report  —  DI-MGMT-81 466A*  5  Formats  (WBS,  Organization,  Baseiine,  staffing  &  Explanation) 
Integrated  Master  Schedule  —DI-MGMT-81 650*  (Mandatory  for  dodevms  contracts) 

Integrated  Baseline  Review  (IBR)  —  Mandatory  for  all  EVMS  contracts 
*See  the  EVMIG  for  CPR  and  IMS  tailoring  guidance.  „„ 
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CONTRACTING— COMPONENTS  OF 
CONTRACT  PRICE 


Contract  Price  =  Cost  +  Profit /Fee 

j 


1 

Direct  Cost 

Direct 

1 

Labor  |  Direct  Material 

Other 

Direct  Cost 

Engineering  Manufacturing  Raw 
Labor  Labor  Material 


- 1 

Indirect  Cost 

I - 1 - 1 

Overhead  G&A  FCCM 


T 

Purchased 
Parts 

Standard 
Comm 

Items  Subcontracts  Interdivisional 
Transfers 


Engineering  Manufacturing  Material 
Support  Support  Handling 


TYPICAL  CONTRACT  TYPE  BY  PHASE 


MSA 

TD 

EMD/ISD 

EMD/SCMPD 

PROD 

CPFF,  FFP 

CPFF,  FFP 

FPI(F),  CPFF,  CPIF 

CPIF,  CPAF 

FPI(F),  FFP 

LEGEND: 

AF  —Award  Fee 
CPAF  — Cost  Plus  Award  Fee 
CPFF — Cost  Plus  Fixed  Fee 
CPIF  — Cost  Plus  Incentive  Fee 
CR  —  Cost  Reimbursement 
EMD  —  Engineering  and  Manufacturing 
Demonstration 

FAR/DFARS  —  Federal  Acquisition 
Regulation/Defense  FAR  Supplement 
FCCM  —  Facilities  Capital  Cost  of  Monies 
FFP  —  Firm  Fixed  Price 


FPI(F)  —  Fixed  Price  Incentive  (Firm  Target) 

G&A — General  and  Administrative  (Expense) 
ISD  —  Integrated  System  Design 
MSA — Materiel  Solution  Analysis 
ODC  —  Other  Direct  Cost 
PROD  —  Production 
R&D  —  Research  and  Development 
SCMPD  —  System  Capability  and  Manufacturing 
Process  Demonstration 
TD  —  Technology  Development 


TYPES  OF  CONTRACTS 


Cost  Family  —  Appropriate  when  product  not  well  defined;  high  risk; 
contractor  provides  best  efforts;  government  pays  ail  allowable  costs.  Fee 
varies  by  type. 

Cost  Plus  Fixed  Fee  —  Fee  same  regardless  of  actual  cost  outcome. 

Cost  Plus  Incentive  Fee  —  Actual  fee  earned  computed  by  applying  share 
ratio  to  over/under  run,  subject  to  min/max  fee  limits. 

Fixed  Price  Family  —  Product  well  defined,  low  risk;  contractor  must  deliver 
product. 

Firm  Fixed  Price  —  Price  fixed  regardless  of  actual  cost  incurred. 

Fixed  Price  Incentive  (Firm  Target)  —  Final  price  computed  by  applying 
share  ratio  to  over/underrun,  subject  to  ceiling  price  limitation. 

Award  Fee  —  Either  stand-alone  Cost  Plus  Award  Fee  or  combined  with  cost 
or  fixed  price  types.  Award  Fee  unilaterally  determined  by  government  based 
on  subjective  evaluation  of  contractor’s  performance. 

Fee  Limits:  Cost  Plus  Fixed  Fee  —  Fee  limited  to  1 5%  for  R&D;  1 0%  for 
production  and  services.  No  statutory  or  FAR/DFARS  regulatory  limitation  on 
other  contract  types. 
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CONTRACT  TYPE  FEATURES 


FIXED 

COST 

PRICE 

REIMBURSEMENT 

Promise 

Deiivery 

Best  Efforts 

Contractor  Risk 

High 

Low 

Cash  Flow 

Deiivery 

As  Incurred 

Progress  Payments  % 

75/90/95 

N/A 

Administration 

Low 

High 

Fee  Limit  % 

None 

15/10/6  on  CPFF 

FEE 

Cost  Plus  Fixed  Fee  (CPFF) 

FIXED 

(SHARE  100/0) 

FEE 

I 

I 

I 

I 

^ 

COST  ESTIMATED  COST 

PRICE  =  COST  +  FIXED  FEE 

•  Risk  Highest  to  the  Government 

•  Obtains  Fee  Regardless  of  Cost 

Cost  Plus  Incentive  Fee  (CPIF) 


(Target)  PRICE  =  (Target)  COST  +  (Target)  FEE 

•All  Reasonable  Cost  Paid 
•  Shared  Risk  Between  Min/Max  Fee 
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CONTRACT  TYPE  FEATURES 

(Continued) 


Fixed  Price  Incentive  (Firm  Target)  (FPI(F)) 


(Target)  PRICE  =  (Target)  COST  +  (Target)  PROFIT 

Point  of  Total 


Assumption  = 
(PTA) 


_  CEILING  PRICE  -  TARGET  PRICE 


GOVERNMENT  SHARE 


Target  Cost 


Firm  Fixed  Price  (FFP) 

PROFIT 

- \  0/100  SHARE 

l\ 

I  \ 

l\ 

COST  \ 

PRICE  =  COST  +  PROFIT 
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PRE-SOLICITATION  PROCESS 


POST-SOLICITATION  PROCESS 
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OTHER  WAYS  TO  BUY 

•  GSA  Multiple  Award  Schedules  (MAS) 

-  General  Services  Administration  contracts  for  both  products  and 
services — avaiiable  to  aii  agencies. 

•  Government-Wide  Agency  Contracts  (GWACs) 

-  Similar  to  MAS,  but  more  restricted  in  products  and  services  avaiiabie. 

•  Indefinite  Delivery/Indefinite  Quantity  Contracts 

-  Task  orders  (services)  and  deiivery  orders  (products)  issued  under 
omnibus  umbreila  contract. 

•  OtherTransactions  (OT) 

-  Defined:  Vehicles  used  for  basic,  applied,  and  advanced  research 
projects  and  prototype  development.  OTs  are  not  contracts,  grants, 
or  cooperative  agreements. 

-  Objective:  Attract  commercial  companies  and  consortia  that 
historically  have  not  done  business  with  the  Department  of 
Defense  because  of  statutory  and/or  regulatory  requirements.  OTs 
are  not  subject  to  the  Federal  Acquisition  Regulation.  Designed  to 
increase  DoD  access  to  dual-use  technologies. 

-  Research  Projects: 

♦  Where  practical,  government  cost  share  should  not  exceed  cost 
share  of  other  parties. 

♦  Use  OT  when  standard  contract,  grant,  or  cooperative  agreement  is 
not  appropriate. 

-Prototype  Projects: 

♦  Must  be  directly  relevant  to  weapons  or  weapon  systems 
proposed  to  be  acquired  or  developed  by  DoD. 

-  Constraints: 

♦  At  least  one  nontraditional  contractor  participating. 

♦  If  no  nontraditional  contractor  participates,  1/3  of  cost  paid  by 
parties  other  than  federal  government  or  senior  procurement 
executive  justifies  transaction. 

-  OT  Guide  for  Prototype  Projects,  January  2001. 
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CONTRACTOR  PROFITABILITY  RATIOS 


The  basic  concept  of  profitability  ratios  is  to  measure  net  income  against  revenue  or  against 
the  investment  required  to  produce  it.  There  are  three  principal  profitability  ratios  with  which  you 
should  be  familiar.  They  are: 

1 .  Return  on  Sales,  which  shows  what  percentage  of  dollars  are  left  after  the  company 
has  paid  for  all  costs,  interest,  and  taxes.  It  is  expressed  as: 


Return  on  Sales 


Net  Income 
Sales 


2.  Return  on  Total  Assets,  which  looks  at  the  efficiency  with  which  management  has  used 

its  resources,  the  company’s  assets,  to  generate  income.  It  is  computed  as: 


ROA  = 

Total  Assets 

As  noted,  Return  on  Assets  addresses  how  well  management  utilizes  the  assets  of  the  firm  in 
generating  income.  The  ROA  formula  reflects  the  combined  result  of  Return  on  Sales  and  the 
total  asset  turnover  ratio  (total  sales/total  assets),  broken  down  as  follows: 


_  Net  Income  X  Total  Sales 

Total  Sales  Total  Assets 

3.  Return  on  Stockholders’  Equity  measures  the  rate  of  return  on  the  owners' 
investment— their  equity  in  the  company.  This  is  also  known  as  Return  on  Equity; 

PQg  _  Net  Income 

Stockholders’  Equity 

ROE  can  also  be  broken  into  two  components:  return  on  assets  and  financial  leverage  (a  ratio 
reflecting  the  relationship  of  creditor  to  owner  financing— expressed  as  total  assets/stockholders 
equity).  This  is  shown  by: 


PQg  _  Net  Income  X  Total  Assets 

Total  Assets  Stockholders’  Equity 

These  profitability  ratios  give  three  different  viewpoints  concerning  the  “bottom  line”  on  the  income 
statement— how  much  net  profit  is  being  made  on  each  sale,  how  much  is  being  made  for  the 
assets  that  are  employed,  and  how  much  is  being  made  for  the  company  owners.  Contractor 
profitability  ratios  for  the  aerospace/defense  industry  for  the  period  of  1 980  to  date  are  shown 
on  page  38. 

From  an  owner’s  perspective,  another  profitability  ratio  you  may  be  aware  of  is  Earnings  Per 
Share: 

^pg  _  _ Net  Income _ 

Number  of  Shares  of  Common  Stock  Outstanding 


LEGEND: 

EPS— Earnings  Per  Share 

ROA— Return  on  Assets 

ROE— Return  on  Equity 
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AEROSPACE/DEFENSE  INDUSTRY 
CONTRACTOR  PROFITABILITY  RATIOS 

Aerospace/Defense  Industry  Contractor 
Profitability  Ratios 


Return 

On  Sales 
(Nl/S) 

Asset 

Turnover 

(S/TA) 

Return 

On  Assets 
(NI/TA) 

Financial 

Leverage 

(TA/SE) 

Return 
On  Equity 
(Ni/SE) 

1980 

4.3 

1.21 

5.2 

3.08 

16.0 

1981 

4.4 

1.18 

5.2 

3.06 

16.0 

1982 

3.3 

1.12 

3.7 

3.24 

12.0 

1983 

3.5 

1.17 

4.1 

2.98 

12.1 

1984 

4.1 

1.15 

4.7 

3.00 

14.1 

1985 

3.1 

1.13 

3.6 

3.17 

11.1 

1986 

2.8 

1.07 

3.1 

3.13 

9.4 

1987 

4.1 

1.07 

4.4 

3.32 

14.6 

1988 

4.3 

1.02 

4.4 

3.39 

14.9 

1989 

3.3 

1.00 

3.3 

3.24 

10.7 

1990 

3.4 

1.00 

3.4 

3.38 

11.5 

1991 

1.8 

1.06 

1.9 

3.21 

6.1 

1992 

-1.4 

0.86 

-1.2 

4.33 

-5.2 

1993 

3.6 

0.97 

3.5 

3.80 

13.2 

1994 

4.7 

0.92 

4.3 

3.44 

14.8 

1995 

3.8 

0.92 

3.5 

3.17 

11.1 

1996 

5.6 

0.91 

5.1 

3.35 

17.1 

1997 

5.2 

0.92 

4.8 

3.60 

17.3 

1998 

5.0 

0.96 

4.8 

3.73 

18.0 

1999 

6.5 

0.95 

6.2 

3.52 

21.8 

2000 

4.7 

0.91 

4.3 

3.30 

14.2 

2001 

3.9 

0.92 

3.6 

3.22 

11.6 

2002 

4.1 

0.90 

3.7 

3.16 

11.7 

2003 

3.1 

0.84 

2.6 

3.81 

9.9 

AVERAGE 

3.8 

1.0 

3.8 

3.4 

12.7 

Source:  Aerospace  Industries  Association. 


LEGEND: 

Nl/S— Net  Income/Sales 

STA— SalesTotal  Assets 

NI/SE— Net  income/stockholders’  Equity 
NITA— Net  IncomeTotal  Assets 

TA/SE— Total  Assets/Stockholders'  Equity 
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LOCKHEED  MARTIN  CORP.— DUPONT  FORMULA  ANALYSIS  (AN  EXAMPLE) 

Fiscal  Returnon  Total  Asset  Return  Return  Financial  Return 

Year _ Sales  %  X _ Turnover _ =  on  Assets  % _  on  Assets  %  X  Leverage  =  on  Equity  % 
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X 
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CASH  CYCLE 


Contract 

award 


Cash 

received 


/ 


Raw  material 
inventory 


Accounts 

receivable 


\ 


I 


Work  in  process 


Sale 

(DD  250) 


inventory 

/ 

Finished  goods 
inventory 


\ 

Wages 

payable 

I 


Accounts 

payable 


I 


Cash 

disbursed 


Cash 

disbursed 


CONTRACTOR  FINANCING  AND  PAYMENTS 


FINANCING  (External* 


PAYMENTS 


Commercial 

Noncommercial 

Commercial 

Noncommercial 

—  Govt,  specified 

—  Offer  or  proposal 

—  Interim 

—  Advance 

—  Private 

•  Trade  Credit 

•  Bank  Credit 

—  Revolving  Credit 

—  Delivery 

—  Periodic 

—  Partial 

—  Term  Loan 

—  Government 

•  For  Noncommercial 

—  Progress  Payments 

•  Performance-based 

•  Cost  Incurred-based 

•  %  Complete 
-Unusual  Progress 

Payments 

-Assignment  of  Claims 
-Guaranteed  Loans 
-Advance  Payments 


Internal  Contractor  Financing  —  Retained  Earnings 
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DIRECT  AND  INDIRECT  COSTS 


Direct  costs 

•  Costs  that  can  be 
traced  to  a  single 
contract 

•  Examples:  material 
and  labor  to  assemble 
an  aircraft 


Indirect  costs 

•  Costs  that  can’t  be  traced 
to  a  single  contract 
because  they  are 
associated  with 
multiple  contracts 

•  Example:  Electricity  for 
the  company’s  facilities 


ASSIGNING  INDIRECT  COSTS 


Indirect  costs  are 
assigned  to 
contracts  using 
indirect  rate(s). 


Note:  See  DAU  Indirect-Cost  Management  Guide  at 
<www.dau.mil/pubs/gdbks/icmguide.asp>. 


Calculation  of  Indirect  Rates 


INDIRECT  RATE  = 


Indirect  Cost  Pool 
Allocation  Base 
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LIFE  CYCLE  OF  INDIRECT  COST  RATES 


t  t  I 


BIDDING  ON 
CONTRACTS 

PAYING 

CONTRACTS 

ADJUSTING 
PAYMENT  AND 
CLOSING 
CONTRACTS 

CONTRACTOR’S  COST  PROPOSAL 
EXAMPLE 


Direct  material 

Material  handling  10% 

$  40,000 
4,000 

Direct  engineering  labor 

Engineering  overhead  100% 

6,000 

6,000 

Direct  manufacturing  labor 

Manufacturing  overhead  150% 

12,000 

18,000 

Other  direct  costs 

Subtotal 

6,000 

92,000 

General  and  administrative  25% 

Total  cost 

23,000 

115,000 

Profit  15% 

Cost  of  money  for  facilities  capital 

17,250 

1,500 

Price 

($133750 
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CONTRACTOR  BUSINESS  PLANNING 
PROCESS  OUTPUTS 


TOP  MANAGEMENT  OVERHEAD 
GOALS,  OBJECTIVES,  FORECAST 
STRATEGIES 


PRODUCTION  AND 
ENGINEERING  PLAN 


MANPOWER  PLAN 


ANNUAL 

OPERATING  PLAN 


IR&D/B&P  PLAN 


CAPITAL 

INVESTMENT  PLAN 


MASTER  DELIVERY 
SCHEDULE 


BUSINESS  BASE 
SALES  FORECAST 


LEGEND: 

B&P— Bid  of  Proposal 

IR&D— Independent  Research  and  Development 
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LIFE  CYCLE  LOGISTICS 

The  planning,  development,  implementation,  and  management 
of  a  comprehensive,  affordable,  and  effective  product  support 
strategy,  within  Total  Life  Cycle  Systems  Management.  Life  Cycle 
Logistics  encompasses  the  entire  system’s  life  cycle,  including 
acquisition  (design,  develop,  test,  produce,  and  deploy),  sustainment 
(operations  and  support),  and  disposal. 


PRINCIPAL  LIFE  CYCLE  LOGISTICS 
GOALS/OBJECTIVES 

•  Goal  1 :  Define  supportability  objectives  and  establish  life  cycle 
sustainment  metrics. 

•  Goal  2:  Influence  system  design  for  affordable  system  operational 
effectiveness. 

•  Goal  3:  Design  and  develop  the  support  system  utilizing 
performance-based  life  cycle  product  support. 

•  Goal  4:  Acquire  and  deploy  efficient  and  effective  product  support 
to  maintain  the  readiness  and  operational  capability  of  the  system. 

•  Goal  5;  Continuously  improve  readiness  and  affordability  through 
enhanced  life  cycle  management. 


GOAL  1 :  LIFE  CYCLE  SUSTAINMENT  METRICS 

Availability  Key  Performance  Parameter  (KPP)  consists  of  two  components: 
Materiel  Availability  Operational  Availability 


Materiel  Availability: 

•  Development  of  the  Materiel  Availability  metric  is  a  Program 
Manager  responsibility 

•  Addresses  the  total  population  of  end 
items  planned  for  operational  use 

•  A  measure  of  the  percentage  of  the 
total  inventory  operationally  capable  of 
performing  an  assigned  mission 

•  Formula:  Number  of  End  Items 
Operational  ^  Total  Population  of  End 
Items 
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Operational  Availability  (Aq): 

•  Development  of  the  Operational  Availability  metric  is  a  Requirements 
Manager  responsibility 

•  Indicates  the  percentage  of  time  that  a  system  or  group  of  systems 
within  a  unit  are  operationally  capable 

•  Formula:  Uptime  (Uptime  +  Downtime) 

Reliability  Key  System  Attribute  (KSA) 

•  Development  of  the  Reliability  metric  is  a  Requirements  Manager 
responsibility 

•  Measure  of  the  probability  that  system  will  perform  without  failure 
over  a  specific  interval 

•  Must  support  both  Availability  metrics 

Ownership  Cost  Key  System  Attribute  (KSA) 

•  Development  of  the  Ownership  Cost  metric  is  a  Program  Manager 
responsibility 

•  O&S  value  should  cover  the  planned  life  cycle  timeframe 

•  Balance  to  the  sustainment  solution  by  ensuring  O&S  costs  are 
considered 

•  Use  the  CAIG  Cost  Estimating  Structure 

Mean  Downtime  Sustainment  Outcome  Metric 

•  Average  total  Downtime  required  to  restore  an  asset  to  its  full 
operational  capabilities 

•  “Umbrella”  metric  that  captures  all  causes  of  system  Downtime 

•  Formula:  Total  Downtime  for  All  Failures  Total  Number  of  Failures 
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GOAL  2:  AFFORDABLE  SYSTEM  OPERATIONAL 
EFFECTIVENESS  (SOE) 


Capabilities 


Performance 


Reliability 


I  Maintainability 
Support  Features 


Design 


Supportability 


Product 

Support 

Package 

Public  Private 
Support 
Infrastructure 

Logistics 

Operations 


Effectiveness 


Efficiency 


Affordable 

System 

Operational 

Effectiveness 


Life  Cycle  Cost/Total  Ownership  Cost 


To  achieve  Affordable  System  Operational  Effectiveness  (SOE), 
the  PM  should  design  for  the  optimal  balance  between  technical 
performance  (including  Reliability,  Availability,  and  Maintainability 
(RAM)),  categories  of  ownership  cost,  schedule,  and  process 
efficiencies.  A  development  program  that  targets  only  some 
categories  of  technical  performance  capability;  or  fails  to  optimize 
system  RAM  technical  performance,  risks  financial  burden  during 
operations  and  support. 


GOAL  3:  DOD  PRODUCT  SUPPORT  STRATEGY 
PROCESS  MODEL 

•  A  product  support  strategy  encompasses  the  means  by  which  de¬ 
fense  system  sustainment  is  to  be  accomplished. 

•  The  Life-Cycle  Product  Support  Strategy  Process  Model  represents 
the  major  activities  required  to  implement,  manage,  evaluate,  and 
refine  product  support  over  the  system  life  cycle.  It  is  not  a  one¬ 
time  process,  but  rather  a  continuing,  iterative  process  in  which  the 
sustainment  of  a  system  (or  systems)  is  adapted  and  evolved  to 
optimally  support  the  needs  and  requirements  of  the  warfighter  in  an 
affordable  and  effective  manner. 
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Integrated  Warfighter 
Requirements  Jj 


^  Form  the  Product  1 
Support  Management 
^  IPT  -A 


Implement  & 
Assess 


Establish/Refine 
Product  Support 
^Agreement^ 


Baseline 
the  System 


Identify/Refine 
Performance 
.  Outcomes  - 


Identify/Refine 
Financial  Enablers 


Identify  Product 
Support  Provider(s) 


Business  Case 
Analysis 


Integrated  Commercial  +  Government 


Designate  Product 
Support  lntegrator(sj 


Product  Support 
Value  Analysis 


Determine  Support 
Method(s)  ^ 


12-Step  Process  Model 

<https://acc.dau.mil/pbl> 


GOAL  4:  INTEGRATED  PRODUCT  SUPPORT 
ELEMENTS 


INTEGRATED  PRODUCT  SUPPORT 


Product  Support  Management 


(See  next  page  for  details) 
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Product  Support  Management:  Plan,  manage,  and  fund  weapon  sys¬ 
tem  product  support  across  all  Integrated  Product  Support  (IPS) 
Elements. 

Design  Interface:  Design  interface  is  the  integration  of  the  quantitative 
design  characteristics  of  systems  engineering  (reliability,  maintain¬ 
ability,  etc.)  with  the  functional  logistics  elements  (i.e.,  integrated 
product  support  elements)  described  below. 

Sustaining  Engineering:  Those  technical  tasks  (engineering  and  logis¬ 
tics  investigations  and  analyses)  to  ensure  continued  operation  and 
maintenance  of  a  system  with  managed  (i.e.,  known)  risk. 

Supply  Support:  Management  actions,  procedures,  and  techniques 
necessary  to  determine  requirements  to  acquire,  catalog,  receive, 
store,  transfer,  issue  and  dispose  of  spares,  repair  parts,  and 
supplies. 

Maintenance  Planning  and  Management:  Establishes  maintenance 
concepts  and  requirements  for  the  life  of  the  system  for  both  hard¬ 
ware  and  software. 

Packaging,  Handling,  Storage,  and  Transportation  (PHS&T):  The 

combination  of  resources,  processes,  procedures,  design,  consid¬ 
erations,  and  methods  to  ensure  that  all  system,  equipment,  and 
support  items  are  preserved,  packaged,  handled,  and  transported 
properly. 

Technical  Data:  Represents  recorded  information  of  scientific  or  techni¬ 
cal  nature,  regardless  of  form  or  character  (such  as  equipment 
technical  manuals  and  engineering  drawings),  engineering  data, 
specifications,  standards,  and  Data  Item  Descriptions  (DID). 

Support  Equipment:  Consists  of  all  equipment  (mobile  or  fixed)  re¬ 
quired  to  support  the  operation  and  maintenance  of  a  system. 

Training  and  Training  Support:  The  policy,  processes,  procedures, 
techniques.  Training  Aids  Devices  Simulators  and  Simulations 
(TADSS),  planning  and  provisioning  for  the  training  base  including 
equipment  to  acquire,  operate,  maintain,  and  support  a  system. 

Manpower  and  Personnel:  Involves  the  identification  and  acquisition  of 
personnel  (military  and  civilian)  with  the  skills  and  grades  required 
to  operate,  maintain,  and  support  systems  over  their  lifetime. 

Facilities  and  Infrastructure:  Consists  of  the  permanent  and 
semi-permanent  real  property  assets  required  to  support  a 
system,  including  studies  to  define  types  of  facilities  or  facility 
improvements,  location,  space  needs,  environmental  and  security 
requirements,  and  equipment. 

Computer  Resources:  Encompass  the  facilities,  hardware,  software, 
documentation,  manpower,  and  personnel  needed  to  operate  and 
support  mission  critical  computer  hardware/software  systems. 
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GOAL  5:  PRODUCT  SUPPORT  BUSINESS  MODEL 

(PSBM) 


Recomme/ic/af/on;  Adopt  a  “product  support  business  model”  that  drives  cost-effective  performance 
and  capability  forthe  warfighter  across  the  weapon  system  life  cycle  and  enables  the  most 
advantageous  use  of  an  integrated  defense  industrial  base. 


•  The  PSBM  encompasses  the  overall  strategy  for  product  support 
planning,  implementation,  management,  and  measurement  over  the 
life  cycle  of  a  weapon  system  component,  subsystem,  or  platform. 

•  A  Product  Support  Manager  (PSM)  will  provide  product  support 
subject  matter  expertise  to  the  PM  for  execution  of  the  PM’s 
duties  as  the  Total  Life  Cycle  Systems  Manager.  This  PSM  will  be 
designated  as  a  key  leadership  position  (KLP)  for  all  Major  Defense 
Acquisition  Programs  and  designated  a  critical  acquisition  position 
(CAP)  for  all  other  major  weapon  systems.  Product  Support 
Integrators  (PSIs)  are  charged  with  integrating  all  sources  of  product 
support,  both  public  and  private,  defined  within  the  scope  of  a 
product  support  arrangement. 
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THE  SUSTAINMENT  CHART 

The  “Sustainment  Chart”  is  one  tooi  the  PM  may  use  for  key  decision-making 
because  it  readiiy  identifies  a  weapon  system’s  product  support  business 
modei  and  captures  its  operating  and  support  costs  and  operationai  metrics 
data.  PMs  are  required  to  use  the  sustainment  chart  to  report  status  of  sus¬ 
tainment  pianning  at  OIPT  and  DAB  meetings. 


SAMPLE  PROGRAM:  “ABC" 


I  Date: 


Product  Support  Strategy 


Metrics  Data 


Sustainment  Approach 

•  Current  (initial  CLS  covering  total  system) 

■  Future  (sub-^stem  based  PBL  contracts) 

Issues 

•  Shortfall  in  O&M  funding  in  FYDP 

•  Reliability  and  availability  estimates  are  below  goals 

•  LCSP  requires  update  before  DAB 

Resolution 

■  POM  requestforO&M  restoration  submitted 

■  Reliability  improvementplan  with  clear  RAM  goals  up  for 
final  signature 

•  LCSP  in  draft 


Today 

MSB  I  MSC  IOC  FRP 


Sustainment  Schedule 

FOC  Sustainment 


4  B^A 


PBL  Recompete 


^  LRIP  ConttiKt  Awaitl  ^  Avionics  PBL 

^  CLSStait  I  I  I  I  ^  PBL  Recompeie 


^  Blended  Paiiiieisliip 
Startup 


LOGISTICS  MANAGEMENT 
COMMUNITY  OF  PRACTICE  (LOG  COP) 

Transforming  the  IVay  We  Work 

Where  YOU  can  .... 

Find  Helpful  Tools  and  Templates 

•  Latest  PBL  Resources 

•  Supportability  Best  Practices 

•  Contracting  Lessons  Learned 

Get  Ahead  In  YOUR  Career 

•  Logistics  Training  and  Education 

•  Latest  OSD  Policy  and  Direction 

•  Logistics  Conferences/Events 

•  Link  to  Top  DoD  Websites 

Connect  With  Professionals 

•  Share  Experiences  and  Ideas 

•  Start  and  Join  Discussions 


<http://acc.dau.mil/log> 

Locate  DoD  and  Industry  Experts 
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DEFENSE  ACQUISITION  GUIDEBOOK 
LIFE  CYCLE  LOGISTICS 

<https://acc.dau.mil/dag_5> 


LIFE  CYCLE  LOGISTICS  REFERENCES  &  GUIDANCE 

•  Defense  Acquisition  Guidebook  (DAG)  Chapter  5  -  https://acc.dau.mil/dag_5 

•  DAU  Logistics  Community  of  Practice  (LOG  CoP)  -  https://acc.dau.mil/log 

•  Product  Support  Manager  (PSM)  Homepage  -  https://acc.dau.mil/psm 

•  PSM  ACQuipedia  Site  -  https://acc.dau. mil/CommunityBrowser.aspx?id=375980 

•  Performance  Based  Life  Cycle  Product  Support  (PBL)  Toolkit  -  https://acc.dau.mil/pbl 

•  Life  Cycle  Sustainment  Plan  (LCSP)  -  https://acc.dau.mil/lcsp 

•  DoD  Logistics  Human  Capital  Strategy  (HCS)  -  http://www.acq.osd.mil/log/sci/hcs.html 

•  Life  Cycle  Logistics  ACQuipedia  Repository  -  https://acquipedia.dau.mil/log_lcic.aspx 

•  Life  Cycle  Logistics  Blog  -  https://dap.dau.mil/career/log/blogs/default.aspx 

•  Recommended  Reading  List  -  https://acc.dau.mil/CommunityBrowser. 
aspx?id=383460 

•  Logistics  Career  Field  Gateway  -  https://dap.dau.mil/career/log 

•  DAU  Life  Cycle  Logistics  Media  Library  -  http://www.dau.mil/mpi/default.html 

•  Integrated  Defense  AT&L  Life  Cycle  Framework  Chart  -  https://ilc.dau.mil/ 

•  Army  Life  Cycle  Logistics  Framework  Chart  -  https://acc.dau.mil/logsa 

•  Joint  Life  Cycle  Logistics  Framework  Chart  -  Will  be  posted  on  the  LOG  CoP 

•  Product  Support  Manager’s  (PSM)  Guidebook  -  Will  be  posted  on  the  LOG  CoP 

•  Business  Case  Analysis  (BCA)  Guidebook  -  Will  be  posted  on  the  LOG  CoP 

•  Life  Cycle  Logistics  Guidebook  -  Will  be  posted  on  the  LOG  CoP 
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PROGRAM  OFFICE  ORGANIZATION  STRUCTURES 

(Examples) 

“Traditional”  or  Functional  Structure 


P 

M  1 

Staff  1 

Engineering  Business/  | 
Finance  | 

Logistics  | 

Production 

Staff 

Note:  Functional  divisions  shown  are  notional. 


“Pure”  Product  Structure 


Product/ 
Project 
Manager 
System  A 


Functional 

Divisions 

- 1 - r - 

Engrl  Log  I  Bus 


Product/ 
Project 
Manager 
System  C 

- 1  Staff  1 

Functional 

Divisions 

Engrl  Log  1  Bus 

Note:  Functional  divisions  shown  are  notional. 


LEGEND: 

Engr— Engineering 

Log— Logistics 

Bus— Business 
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PROGRAM  OFFICE  ORGANIZATION  STRUCTURES 

(Continued) 

Matrix  Structure 


Component  Head 


Functional  Directors  | 

III  1 

Engineering  | 

Logistics  1 

Production 

1  Bus/FIn  1 

PM  _ I _ I _ I _ !_ 

Program  C  1111 

^  I  Staff]  111  1 

NOTE:  Functional  divisions  shown  are  notional,  t  t  t  t 


Integrated  Product  Teams 


Level  1 
IPT 


Level  2 
IPTs 


Level  3 
IPTs 


Program  Manager 

DPM 

APMs 

Log  1  Eng  Test  1  Prod  i  Bus/Fin  :  Contracts 

Primary 

Vehicle 


Note  1 


Systems 

Engineering 


Logistics 


Test  & 
Evaluation 


Note  2 


Frame 

Suspension 

Engine/Drive 

Fire 

&  Steering 

Train 

Control 

NOTE  1:  Functional  titles  shown  are  notional. 
NOTE  2:  IPTs  often  align  with  WBS  elements. 
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THE  ROLE  OF  MANUFACTURING  IN  THE 
ACQUISITION  PROCESS 


CURRENT  DoD  5000  PROCESS 


A  A 


A 


MSA 

TD 

INTEGRA¬ 

TION 

DEMON¬ 

STRATION 

LRIP 

FULL 

RATE 

SUSTAINMENT 

RDT&E 

PRODUCTION 

•  Influence  the  Design 

•  Execute  the  Manufacturing  Pian 

Process 

-Refiect  Design  Intent 

•  Prepare  for  Production 

-Repeatable  Processes 

•  Continuous  Process  Improvement 

EMD  EXIT  REQUIREMENTS 

NET  RESULT 

•  Manufacturing  processes  have 

•  Uniform,  Defect-Free 

been  demonstrated  in  a  pilot  line 

Product 

environment 

•  Consistent  Performance 

•  No  significant  manufacturing  risks 

•  Lower  Cost 

COMMON  PRODUCTION  RISKS  THAT  GREATLY 
IMPACT  COST,  SCHEDULE,  AND  PERFORMANCE 

•  Unstable  requirements/engineering  changes 

•  Unstable  production  rates  and  quantities 

•  Insufficient  process  proofing 

•  Insufficient  materials  characterization 

•  Changes  In  proven  materials,  processes,  subcontractors, 
vendors,  components 

•  Produciblllty 

•  Configuration  management 

•  Subcontractor  management 

•  Special  tooling 

•  Special  test  equipment 
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PRODUCIBILITY 


DEFINITION: 

The  measure  of  relative  ease  of  manufacturing  a  product.  The  product 
should  be  easily  and  economically  fabricated,  assembled,  inspected, 
and  tested  with  high  quality  on  the  first  attempt  that  meets  perfor¬ 
mance  thresholds. 

PRODUCIBILITY  ISSUES: 

•  Design  engineering,  NOT  manufacturing,  is  the  technical  group 
responsible  for  producibility.  Program  offices  and  design  engineers 
often  dislike  producibility  because  it  usually  requires  performance 
functionality  sacrifices  (especially  if  cost  is  a  set  value,  i.e.,  CAIV). 

•  Many  design  engineers  do  not  have  proper  training  or  experience  in 
designing  for  producibility.  Manufacturing  facilities  must  be  explicitly 
recognized  as  a  major  design  constraint.  This  includes  process 
capabilities  and  rate  capabilities  at  each  facility. 

The  PM  is  responsible  for  Producibility 


Producibility 

(see  Defense  Acquisition  Guidebook) 

•  Producibility:  degree  to  which  system  design  facilitates  timely,  afford¬ 
able,  optimum-quality  manufacture,  assembly,  and  delivery  of  system. 

•  Producible  system  design  should  be  a  development  priority. 

•  Design  engineering  efforts  concurrently  develop: 

—  Producible  and  testable  design; 

—  Capable  manufacturing  processes;  and 

—  Necessary  process  controls  to: 

♦  Meet  requirements,  and 

♦  Minimize  manufacturing  costs. 

•  PM  should  use  existing  manufacturing  processes  whenever  possible. 

•  When  design  requires  new  manufacturing  capabilities,  PM  needs  to 
consider  process  flexibility  (e.g.,  rate  and  configuration  insensitivity). 

•  Full-rate  production  necessitates: 

—  Stable  systems  design; 

—  Proven  manufacturing  processes;  and 

—  Available  production  facilities  and  equipment. 
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Quality  Management  Systems 

(see  Defense  Acquisition  Guidebook) 

•  The  PM  should  allow  contractors  to  define  and  use  their  preferred 
quality  management  system  that  meets  required  program  support 
capabilities. 

•  The  PM  will  not  require  International  Standards  Organization 
(ISO)  registration  of  a  supplier’s  quality  system  since  there  have 
been  instances  where  ISO  9001 -registered  supplier  products  were 
deficient  or  life-threatening. 

•  Contractor’s  quality  management  system  should  be  capable  of  the 
following  key  activities: 

-  Monitor,  measure,  analyze,  control,  and  improve  processes; 

-  Reduce  product  variation; 

-  Measure/verify  product  conformity; 

-  Establish  mechanisms  for  field  product  performance  feedback;  and 

-  Implement  an  effective  root-cause  analysis  and  corrective  action 
system. 

NOTES:  ISO  9000  Series  International  Quality  Standard  is  considered  a  Basic  Quality  system,  but  the 
focus  is  still  on  “Document  what  you  do.  Do  what  you  document.” 

Advanced  Quality  Systems  (AQS),  such  as  the  new  SAE  AS91 OOB  Aerospace  industries'  quality  standard, 
focus  on  achieving  customer  satisfaction  via  use  of  key  characteristics  identification  and  control,  variation 
reduction  of  key  characteristics,  flow-down  of  similar  process  control  requirements  to  suppliers,  and  many 
other  advanced  process-oriented  control  and  improvement  techniques. 


Key  Characteristics  and  Variation  Reduction 

GOAL — Minimize  and  control  variation  on  both  key  product  character¬ 
istics  and  corresponding  key  manufacturing  process  characteristics: 

•  Key  Characteristics:  The  features  of  a  material,  process,  or 
part  whose  variation  has  a  significant  influence  on  product  fit, 
performance,  service  life,  or  manufacturability — per  SAE  AS9100B. 

•  Major  Sources  of  Variation:  Insufficient  design  margins,  process 
(manpower,  machinery,  methods,  etc.),  measurement  systems, 
supplier’s  products. 

WHY:  Direct  correlation  between  deviation  from  nominal  value  (i.e., 
variation)  on  key  characteristics  and  product  quality  and  functionality. 

TOOLS:  Quality  Function  Deployment  (QFD),  Design  of  Experiments 
(DOE),  Statistical  Process  Control.  (See  control  chart  on  next  page.), 
Continuous  Process  Improvement  (CPI)/Lean  Six  Sigma  (LSS)  Program. 
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Key  Characteristics  and  Variation  Reduction  (Continued) 


X  (Control  Chart) 


R  (Control  Chart)* 


*Note:  No  lower  control  limit  for  R  chart  for  sample  size  below  7. 

The  X  (X  bar)  and  R  Control  Charts  are  used  to  monitor  manufacturing  processes.  Upper 
or  Lower  Control  Limits  (UCLor  LCL)  are  NOT  design  specification  parameters.  Instead, 
they  are  predicted  boundaries  for  stable  processes,  calculated  using  x  (X  double  bar) 
(average  of  sampled  process  Means),  R  (R  Bar)  (the  average  of  the  sample  Ranges,  which 
are  the  spreads  between  extreme  values  per  sample),  plus  the  selected  data  sample  size 
and  process-keyed  statistical  formulas.  Values  outside  the  UCL  and/or  LCL  indicate  possible 
process  instability,  likely  due  to  uncommon  “special"  causes  of  variation. 

Caution:  A  process  in  control  is  desirable  because  it  is  predictable,  yet  it  could  fail  to  meet 
design  requirements  due  to  inherent  “common”  variation  and/or  because  the  process 
average  isn't  centered  on  the  design  nominal  value. 

Reference:  The  Memory  Jogger™  //;©1 994  by  GOAL/QPC. 
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Production  Readiness  Review  (PRR) 

WHY  WE  DO  THEM 

•  Risk  Management  Tool:  Identify  program  risks  and  issues  and  opportunities  early  and  often 
(small,  incremental,  proactive  vice  big,  single,  reactive). 

•  Assess  capability  of  contractor  (and  subcontractor)  to  deliver  a  product  on  time  and  within  cost 
that  meets  performance  and  quality  requirements. 

•  Assess  actual  contractor  performance  (metrics). 

•  Assess  effectiveness  of  contractor’s  corrective/preventative  actions. 

•  Measure  improvement  of  contractor’s  performance. 

HOWTO  DO  THEM 

•  Write  a  charier  that  the  program  office  and  contractor  both  understand. 

•  Coordinate  with  the  Defense  Contract  Management  Agency  (DCMA)— use  their  capability. 

•  Establish  areas  of  assessment  with  metrics: 

—  Producibility; 

—  Engineering  Change  Orders  (ECO)/design  stability; 

—  Manufacturing  process  control  (key  characteristics); 

—  Cost,  time  of  scrap,  rework,  and  repair; 

—  Tooling  status;  and 

—  Subcontractor  management  (same  metrics  as  listed  above). 

•  Ask  questions,  touch  things,  talk  to  shop  floor  workers: 

—See  what  is  actually  happening  on  the  factory  floor  rather  than  the  conference 
room,  i.e.,  go  see  and  talk  to  the  people  doing  the  work. 

WHEN  TO  DO  THEM 

•  Early  and  often  (see  Defense  Acquisition  Guidebook,  4.3.3.9.3,  Production  Readiness  Reviews). 

•  Concurrently  with  other  technical  reviews,  such  as  the  System  Functional  Review  (SFR), 
Preliminary  Design  Review  (PDR),  and  the  Critical  Design  Review  (CDR). 

•  In  Systems  Integration  and  Systems  Demonstration. 

•  “Final”  PRR  occurs  at  end  of  Systems  Demonstration  (before  Milestone  C). 

•  PRRs  should  be  held  in  LRIP  and  beyond  IF  major  changes  (to  design,  manufacturing 
processes,  rates/quantities,  etc.)  occur  during  LRIP. 


Additional  Manufacturing  Information  Sources 

•  DAU’s  Production,  Quality  and  Manufacturing  Information  Website: 

—  Go  to  <www.dau.mil>;  select  Knowledge  Sharing;  select  Acquisition  Community  Connection; 
then  see  Participate  in  a  Community;  and  select  Production,  Quality  and  Manufacturing. 

—  Contains  references  to  subjects  including  DoD  Manufacturing  Requirements,  and  Best 
Business  Practices,  such  as  Lean  Enterprise,  e-Commerce,  Six  Sigma,  Basic  and 
Advanced  Quality  Systems,  Supply  Chain  Management,  etc. 

•  Best  Manufacturing  Practices  Center  of  Excellence— <www.bmpcoe.org>. 

•  Lean  Aerospace  Initiative  (LAI)^http://lean.mit.edu>. 
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TEST  AND  EVALUATION  (T&E)— TYPES  AND  TASKS 

Developmental  T&E  (DT&E)/Operational  T&E  (OT&E)  Comparisons 


DT&E 

•  Technical  performance  measurement 

•  Developmental  agency  responsible  (PM) 
•Technical  personnel 

•  Limited  test  articles/each  test 
•Controlled  environment 

•  All  types  of  test  articles 

•  Contractor  involved 


OT&E 

•  Operational  effective/suitable 

•  Operational  Test  Agency  (OTA)  responsible 
•“Typical”  user  personnel 

•  Many  test  articles/each  test 

•  “Combat”  environment/threats 

•  “Production  Rep”  test  articles 
•Contractor  may  not  be  allowed  (lOT&E) 


T&E  Required  Before  Going  Beyond  Low  Rate  Initial  Production 

•  Production  Qualification  T&E — Verify  design  article  meets  spec/PM  responsible;  per¬ 
formed  by  contractor  and/or  Govemment/DPRO  assistance  valuable.  Readiness  for  lOT&E. 

•  Live  Fire  T&E  (LFT&E) — Vulnerability  and  Lethality/Developmental  Agency  fund  and 
execute.  DOT&E  oversight,  approval,  and  Congressional  reporting  (LFTE  Report)  for 
ACAT  1,11,  and  selected  programs. 

•  Initial  Operational  T&E  (lOT&E) — Operational  Effectiveness  and  Suitability/Indepen¬ 
dent  Service  OTA  plan  and  manage.  DOTE  oversight,  approval,  and  Congressional 
reporting  (BLRIP  Report)  for  ACAT  I  and  selected  systems. 


T&E  Tasks  and  Events 


Models  and  Simulations  Used  Throughout  the  Acquisition  Process 

To  support  T&E  during: 


Test  Requirements 
•Test  Interfaces 

•  Eval.  Strategy 

•  Systems  Engineering 

•  Design  for  Test 

•  S/W  Human  T&E 
•TESTEMP 

•  Subsystem  T&E 

•  Software  Only  T&E 

System  DT&E 

•  Computer  Software 
Configuration  Item  T&E 

•  Reliability,  Availability, 
and  Maintainability 

•  Supportability 

•  Interoperability 

•  Production  Ouality 
•LFT&E 

•  Certificate  of 
Readiness  for  lOT&E 


REOUIREMENTS  DEVELOPMENT 
ENGINEERING  DESIGN 
FABRICATION  AND  TEST  (BENCH,  LAB) 

INTEGRATION  AND  TEST 
(H/W  IN  THE  LOOP) 

DEVELOPMENT  T&E: 
QUALIFICATION  T&E, 
ACCEPTANCE  T&E,  LFT&E 


OT&E:  EOA,  OA,  lOT&E/OPEVAL 


PRODUCTION  (PAT&E,  PQT&E,  FOT&E) 


System  0T&^ 


DEPLOY  AND  OPERATIONAL  SUPPORT 


•  Effectiveness 

•  Suitability 

•  Acceptance  Test 

•  Manufacturing  Test 


Requirements  Definition 
^T&E  Planning 
Engineering  Design 
Fabrication 
^  Integration 
Systems  DT&E 
OT&E 
Training 
Operations 


•  Data  Collection 

•  Reporting 


Use  Integrated  DT/OT — Single  integrated  contractor/government  DT  and  OT  team; 
shared  test  events  and  test  data;  independent  data  analysis  and  reporting. 

ACAT  I  and  II  Programs — Require  an  independent,  dedicated  lOT&E  to  proceed  beyond  Low 
Rate  Initial  Production  (LRIP). 


Joint  Interoperability  Test  Certification — All  Information  Technology  and  National  Security 
Systems  must  be  evaluated  and  certified  by  the  Joint  Interoperability  Test  Command  (JITC)  prior 
to  (initial  or  updated)  tielding,  and  periodically  during  their  entire  life  -  as  a  minimum,  every  four  (4) 
years. 
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TEST  AND  EVALUATION  NOMOGRAPH 

What  is  a  nomograph?  A  two-dimensional  graphical  representation 
of  the  cumulative  binomial  distribution. 

Why  use  nomograph?  It  enables  a  relatively  simple  solution  to  a 
complex  mathematical  calculation. 

What  does  it  do?  It  allows  you  to  calculate  the  performance  of  an 
item  with  associated  statistical  confidence. 

When  do  you  use  it? 

•  When  your  requirement  includes  a  “Confidence  Level”  with  a  specific 
level  of  performance.  For  example:  THIS  missile  must  hit  THAT  target 
90  percent  of  the  time  with  80  percent  statistical  confidence? 

•  When  the  performance  of  an  item  under  test  can  be  characterized  by 
a  binomial  distribution. 

What  are  the  characteristics  of  a  binomiai  distribution? 

•  Result  of  each  event  (firing)  is  an  independent  from  other  events. 

•  Probability  of  success  of  each  event  is  constant. 

•  Each  event  results  in  a  “success”  or  a  “failure.”  (In  other  words,  there 
are  no  points  for  being  close;  each  event  must  be  scored  as  a  hit  or  a 
miss.) 

What  are  some  exam  pies  of  binomiaiiy  distributed  events? 

•  Coin  flip 

•  Missile  launch 

•  Rocket  firing 

•  Starting  a  car 

BOTTOM  LiNE:  Each  of  these  test  events  must  be  graded  as  “pass”  or 
“fail,”  and  you  must  determine  the  success  criteria  before  the  test  begins. 

The  nomograph  can  be  used  (pre-test)  as  a  test  planning  device  to 
determine  how  many  tests  will  be  necessary  to  verify  that  specified 
performance  has  been  met.  The  nomograph  can  also  be  used  (post¬ 
test)  to  evaluate  test  data. 

NOTE:  There  are  two  axes  on  the  nomograph.  One  axis  is  the  total  number  of  trials.  The  other  axis  in  the 
total  number  of  failures.  Additionally,  the  nomograph  is  non-linear. 
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How  do  you  get  a  solution? 

•  From  the  data,  determine  the  number  of  trials  (total  number  of  coin  flips  or  missile  shots,  etc.)  and  locate  the  appropriate 
line  on  the  nomograph. 

•  Determine  the  number  of  failures  and  locate  the  appropriate  line  on  the  nomograph. 

•  Draw  a  point  at  the  intersection  of  these  two  lines  on  the  nomograph. 

•  Any  straight  line  drawn  through  this  point  is  a  valid  solution  for  the  data  set  used. 

For  example: 

•  Requirement:  Your  missile  must  hit  the  target  at  least  90%  of  the  time,  with  at  least  80%  confidence. 

•  Given:  You  fired  20  missiles  with19  hits  and  1  failure. 

•  What  is  the  statistical  confidence  that  you  will  have  90%  success  in  the  field  with  these  missiles  fired  against  TFIAT  target? 
Answer:  60%  confidence. 

•  Did  you  meet  the  requirement?  NO,  you  achieved  only  60%  confidence  of  hitting  TFIAT  target  90%.  of  the  time,  and  the 
requirement  was  80%.  confidence  or  better.  One  other  way  to  look  at  the  same  data  is  to  say  that  you  did  achieve  90%.  prob¬ 
ability  of  success,  but  you  only  had  60%  confidence  in  this  result;  either  way  you  look  at  it,  you  did  not  meet  the  requirement. 

NOTE:  If  you  had  fired  30  missiles  and  missed  only  1  time,  you  would  have  achieved  the  80%  confidence  along  with  the  required 

90%  performance  level. 
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MODELING  AND  SIMULATION  (M&S)  PLANNING 

•  Access  your  service  centers  for  M&S  expertise 

•  Estabiish  a  simuiation  coordinating  group;  the  EARLiER  the  better 

•  Design  iong-term  M&S  appiications  and  the  integrated  Digitai  Environment  through 
the  acquisition  strategy,  Test  and  Evaluation  Management  Plan  (TEMP),  Source 
Selection  Plan  (SSP) 

•  Create  constructive,  virtual,  or  live  models  and  simulations 

•  CONTINUOUS  PLANNING  PROGRAM  PLANNING 


TEST  AND  EVALUATION  STRATEGYATEMP 


HOW  DO  WE  PLAN?— A  NOTIONAL  APPROACH 

Integrated  Digital  Modeling  and  Simulation  (M&S) 

Environment  (IDE)  Planning  Planning 


•  Establish  a  business  process 
improvement  team 

•  Identify  high  payback  process 
areas 

•  Identify  potential  legacy  systems 
and  data  repositories 

•  Identify  user  base,  including 
remote  sites 

•  Capacity  of  PC  workstations 

•  Bandwidth  of  communication  lines 

•  Where  servers  are/will  be  located 

•  Identify  legacy  system  host 
platforms 


I - — - T 

I  Simulation  Support  Plan  (SSP)  I 
I  (Required  by  Army,  Marine  ' 
I  Corps,  and  Air  Force) 


Identify  high  payback  process  areas 
Identify  potential  legacy  systems, 
Service/Joint-standard  simulations, 
architectures  and  data  repositories 
Identify  where  user  and  simulators 
are/will  be  located 

Determine  capabilities  and  architectures 

of  existing  simulations 

Network  bandwidth  requirements 

IDE  utilization  opportunities 

Interoperability/interface/immersion 

requirements 

Required  capability  cap 

Design  M&S  architectures 

Establish  a  Simulation  and  Verification, 

Validation,  and  Authentication  (SVV&A) 

planning  process 

Establish  long-term  plan,  budget, 

document  and  implement 

Manage,  update,  and  implement  the 

SSP 
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GANTT  CHART 

(Example) 


F 

M 

A 

M 

J 

J 

A 

S 

Activity 


Preliminary  Design 
Design  Analysis 
Define  Interfaces 
Interface  Specs 
Preliminary  Drawings 


Symbol 

Meaning 

1  1 

Planned  activity  schedule 

Status  ot  activity 

1 - 1■■■' 

Forecast  completion  behind  schedule 

1 

Forecast  completion  ahead  of  schedule 

Current 

Date 

NOTE:  There  is  no 
standard  set  of  Gantt 
chart  symbols. 


•  Shows  planned  start  and  finish  dates;  may  also  show  progress. 

•  Depicts  activities  as  horizontal  bars  imposed  over  a  time  line. 

•  Primary  strengths  are  simplicity  and  depicting  overall  project  plan  and  status. 

•  Can  show  dependencies  between  activities  (can  be  difficult  to  read  as  the 
number  of  activities  and  dependencies  between  activities  increases). 


MILESTONE  CHART 

(Example) 


Activity 

IQ  2Q  3Q  4Q  1Q  2Q  3Q  4Q 

1Q  2Q  3Q  4Q 

Contract  Award 

A 

IBR 

A 

SRR 

A 

PDR 

A4 

CDR 

aO 

Current 


Symbol 

Meaning 

A 

Planned  event  completion 

A 

Actual  event  completion 

Actual  completion  behind  schedule 

aO 

Forecast  completion  behind  schedule 

•  Shows  when  key  events  are  scheduled  and  when  they  are  actually 
accomplished. 

•  Primary  strengths  are  simplicity  and  depicting  information  at  the  “big  picture”  level. 

•  Does  not  show  progress  related  to  events  or  dependencies  between  events. 
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NETWORK  SCHEDULES 

Network  Schedules — General 

•  Graphically  portray  dependencies  and  constraints  among  project  activities 
and  the  sequence  in  which  the  activities  occur. 

•  Allows  managers  to  conduct  a  systematic,  disciplined,  and  thorough  review 
of  the  activities  required  to  complete  the  project. 

•  Provides  information  about  early  and  late  start  and  finish  times. 

•  Used  to  determine  the  project’s  critical  path,  and  slack  or  float  in  schedule 
activities. 

•  Generally,  there  are  two  types  of  networks:  Arrow  Diagramming  Method 
(ADM)  and  Precedence  Diagramming  Method  (PDM). 

Arrow  Diagramming  Method  (ADM) 

•  Also  known  as  Activity-on-Arrow  (AOA);  information  about  activities  is  shown 
above/below  the  arrows  connecting  events  in  the  schedules.  Events  are 
usually  shown  as  circles,  squares,  or  rectangles  (see  following  page). 

•  ADM  generally  treats  all  relationships  (see  below)  as  finish-to-start  (i.e.,  first 
activity  must  finish  before  the  next  activity  can  start). 

•  ADM  can  show  other  relationships  (e.g.,  start-to-start,  finish-to-finish) 
through  the  use  of  “dummy”  activities. 


Precedence  Diagramming  Method  (PDM) 

•  Also  known  as  Activity-on-Node  (AON);  information  about  activities  is  shown 
in/on  the  network  nodes.  Nodes  are  usually  shown  as  squares  or  rectangles 
(see  following  page). 

•  Lines  connecting  the  nodes  show  the  relationships  between  the  activities. 

•  PDM  can  show  all  forms  of  schedule  relationships,  including  lead  and  lag 
situations  (see  below). 


Finish-to-Start. 

rXl  r  rEl  Activitv  “A”  must  finish 

1 - 1  1 - 1  before  Activity  “B”  can 

start. 

A 

— 

a 

Start-to-Start. 

Activity  “A”  must  start 
before  Activity  “B”  can 
start. 

A  - 

[1 

Finish-to-Finish. 

Activity  “A”  must  finish 
before  Activity  “B”  can 
finish. 

A 

B 

Start-to-Finish. 

Activity  “A”  must  start 
before  Activity  “B”  can 
finish.  Rareiy  used. 
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SCHEDULE  DEVELOPMENT 
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Network  (Task  Sequence) 


LEAD  TIME  CHART 
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WORKING  DAYS  PRIOR  TO  COMPLETION  (LEAD  TIME) 


LINE  OF  BALANCE  TECHNIQUE 
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PRODUCTION  LEAD  TIME  CHART  22  WORK  DAYS/MONTH 
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ACQUISITION  PROGRAM  BASELINE 

Key  Performance  Parameter  (KPP) — An  Example 


.  Engineering  and  .  Production  and  ^ 

'  Manufacturing  Development  I  Deployment  Phase  ^ 

*ln  this  example,  the  current  estimate  falls  below  the  threshold— this  represents  a  baseline  breach  of 

performance. 

•  Every  program  must  have  an  APB  starting  at  program  initiation  (normally  Milestone  B). 

•  The  APB  reflects  the  threshold  and  objective  values  for  a  minimum  number  of  cost, 
schedule,  and  performance  parameters  that  describe  the  program  over  its  life  cycle. 

•  Cost  thresholds  and  objectives  reflect  major  elements  of  life  cycle  cost  (RDT&E, 
procurement,  PAUC,  APUC,  etc.). 

•  Schedule  thresholds  and  objectives  reflect  critical  events  (milestone  decisions, 
start  of  DT/OT,  first  flight,  IOC,  etc.). 

•  Performance  thresholds  and  objectives  are  key  performance  parameters  (KPPs) 
extracted  verbatim  from  the  CDD/CPD. 

•  The  JROC  requires  KPPs  for  force  protection,  survivability,  sustainment 
(availability),  net-ready,  and  KPPs  traceable  to  the  Joint  Pub  3-0,  Joint 
Operations. 

•  The  MDA  may  add  other  significant  performance  parameters  if  necessary. 

•  The  APB  is  signed  by  PM,  PEO,  and  CAE,  as  appropriate,  and  approved  by  MDA. 


LEGEND: 

APB— Acquisition  Program  Baseline 

MDA— Milestone  Decision  Authority 

APUC— Average  Procurement  Unit  Cost 

MS— Milestone 

CAE— Component  Acquisition  Executive 

PAUC— Program  Acquisition  Unit  Cost  PCDRA— 

CDD— Capabilities  Design  Document 

Post  Critical  Design  Review 

CPD— Capabilities  Production  Document 

Assessment 

DRR— Design  Readiness  Review 

PEO— Program  Executive  Officer 

DT/OT— Development  Test/Operational  Test 

PM— Program  Manager 

FRP  DR— Full-Rate  Production  Design  Review 

RDT&E— Research,  Development, 

IOC— Initial  Operational  Capability 

Test  and  Evaluation 

JROC— Joint  Requirements  Oversight 

Council 
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PROGRAM  MANAGER’S  CHECKLIST  FOR  REVIEW 
OF  TRADEOFF  PLANNING  AND  STUDIES 


1.  ARE  ALL  VIABLE  ALTERNATIVES  BEING 

EXPLORED? 

-  Is  each  alternative  clearly  defined? 

-  Have  the  alternatives  been 
prescreened?  How? 

-  Are  atfordability  limits  established? 

-  Can  all  of  the  screened-out 
alternatives  be  detended? 

2.  ARE  SELECTION  CRITERIA  IDENTIFIED? 

-  Are  all  significant  criteria  identified? 

-  Do  the  criteria  discriminate  among 
alternatives? 

-  Are  the  criteria  measurable? 

-  Have  the  criteria  been  pre-approved? 

3.  IS  THE  CRITERIA  WEIGHTING  SYSTEM 

ACCEPTABLE? 

-  Are  rationales  for  criteria  weights 
explained? 

-  Are  criteria  weights  consistent  with 
guidance? 

-  Are  criteria  weights  consistently 
distributed  in  the  tree? 


4.  ARE  UTILITY  (SCORING)  CRITERIA 

DETERMINED? 

-  Is  defensible  rationale  established  for 
each  criterion? 

-  Are  criteria  developed  from  operational 
measures  of  effectiveness  where 
possible? 

-  Do  all  plans  use  the  same  numerical 
scale? 

-  Is  the  location  of  the  “zero  point” 
explained? 

5.  ARE  EVALUATION  METHODS 

DOCUMENTED? 

-  Are  test  data  reliability  estimates 
(confidence  levels)  incorporated? 

-  Are  models  validated?  When?  By  whom? 

6.  HAS  SENSITIVITY  BEEN  ESTIMATED? 

-  Are  error  ranges  carried  through  with 
worst-on-worst  case  analysis? 

-  Have  the  effects  of  changes  in  the  utility 
curve  shapes  been  examined? 

-  Have  rationales  for  the  limits  been 
developed? 


TECHNICAL  PERFORMANCE  MEASUREMENT 

The  Concept 

Planned  Planned  Tolerance  Band 


Technical  Performance  Measurement  is  a  graphical  depiction  ot  a  product  design  assessment.  It 
displays  values  derived  trom  tests  and  future  estimates  of  essential  performance  parameters.  It  fore¬ 
casts  the  values  to  be  achieved  through  the  planned  technical  program  eftort,  measures  differences 
between  achieved  values  and  determines  the  impact  ot  these  differences  on  system  etfectiveness.  In 
the  DoD,  TPMs  are  typically  related  in  some  way  to  Key  Performance  Parameters  (KPPs). 
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DOD  RISK  MANAGEMENT  PROCESS  MODEL 


Risk  Identification 

1  < -  1 

Risk 

^  1 

Tracking 

Risk  Analysis 


Root  cause  analysis 
of  the  risks  identified 


A  Process,  Not  an  Event 


Risk  Mitigation 
Planning 


Risk  Mitigation 
Implementation 


DoD  Risk  Management  Guide 
August  2006 


ROOT  CAUSE  ANALYSIS 

•  Risk  identification  includes  analysis  to  identify  the  root  causes  of  the 
risks  identified. 

•  Root  causes  are  identified  by  examining  each  WBS  product  and 
process  element  in  terms  of  the  sources  or  areas  of  risk. 

•  An  approach  for  identifying  and  compiling  a  list  of  root  causes  is  to: 

-  list  WBS  product  or  process  elements; 

-  examine  each  in  terms  of  risk  sources  or  areas; 

-  determine  what  could  go  wrong;  and 

-  ask  “why”  multiple  times  until  the  source(s)  is  discovered. 

•  A  common  misconception  and  program  office  practice  concerning 
risk  management  root  cause  analysis  is  to  identify  and  track 
issues  (vice  risks)  and  then  manage  the  consequences  (vice  the 
root  causes).  Risks  should  not  be  confused  with  issues  (realized 
risks).  If  a  root  cause  is  described  in  the  past  tense,  the  root  cause 
has  already  occurred  and  is,  therefore,  an  issue  that  needs  to  be 
resolved  but  not  a  risk! 
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Mitigation  Implementation  through  Integrated  Product  and  Process  Development  (IPPD) 


PROGRAM  SUCCESS  METRICS  MODEL*  SUMMARY 

UH-60  MODEL  SUMMARY  CHART-AN  EXAMPLE 
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WHAT  IS  SYSTEMS  ENGINEERING? 

Systems  Engineering  (SE)  is  an  interdisciplinary  approach  encompassing 
the  entire  technicai  effort  to  evoive  and  verify  an  integrated  and  totai  iife 
cycle  balanced  set  of  system,  people,  and  process  solutions  that  satisfy 
customer  needs.  SE  is  the  integrating  mechanism  across  the  technical 
efforts  related  to  the  development,  manufacturing,  verification,  deployment, 
operations,  support,  disposal  of,  and  user  training  for  systems  and  their 
life  cycle  processes;  and  SE  develops  technical  information  to  support  the 
program  management  decision-making  process.  (DoD  definition) 


SYSTEMS  ENGINEERING  PROCESSES 

DoD  SE  Technical  and  Technical  Management  Processes: 

Technical  Processes:  Stakeholder  Requirements  Definition,  Requirements 
Analysis,  Architecture  Design,  Implementation,  Integration,  Verification,  Validation, 
Transition 

Technical  Management  Processes:  Decision  Analysis,  Technical  Planning, 
Technical  Assessment,  Requirements  Management,  Risk  Management, 
Configuration  Management,  Technical  Data  Management,  Interface  Management 


SYSTEMS  ENGINEERING  POLICY  IN  DOD 

(DoD  5000. 1  and  End  12,  DoD!  5000.02) 

•  Manage  acquisition  programs  through  the  application  of  a  SE  approach  that 
optimizes  total  system  performance  and  minimizes  total  ownership  costs. 

•  SE  Plans  (SEPs)  are  required  for  each  milestone  review.  SEPs 
must  describe  the  overall  technical  approach;  key  risks,  processes, 
resources,  metrics  and  performance  incentives;  the  timing  and  content 
of  technical  reviews  (TRs);  and  item  unique  identification  (lUID)  and 
reliability,  availability,  and  maintainability  (RAM)  summaries. 

•  A  lead  or  chief  systems  engineer  is  required  at  the  program  executive 
officer  (PEO)  level  and  is  responsible  to  the  PEO  for  effective  application 
of  SE  processes  across  the  PEO  portfolio. 

•  Called  out  for  special  emphasis  are: 

— TRs:  Event  driven  and  meeting  SEP  entrance  criteria; 

— Configuration  management:  Required  across  the  life  cycle;  PM  takes 
control  of  product  baseline  after  critical  design  review; 

— Environment,  safety  and  occupational  health:  Required  to  be 
integrated  into  SE  processes; 

— Corrosion  prevention,  and  control:  ACAT  I  programs  require  formal  plan; 
— A  modular,  open-systems  approach  shall  be  employed,  where  feasible; 
— Data  management  strategy:  Required  for  all  ACAT  I  and  II  programs; 

— lUID:  An  implementation  plan  is  required;  and 
— RAM:  A  strategy  is  required  to  include  reliability  growth  program. 
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Implementation 
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LEGEND: 

CDR— Critical  Design  Review  IOC— Initial  Operational  Capability  LRIP— Low-Rate  Initial  Production 

FOC— Full  Operational  Capability  lOT&E— Initial  Operational  Test  and  Eval  PDR— Preliminary  Design  Review 

FRP— Full-Rate  Production  ISD— Integrated  System  Design  SCMPD— System  Capability  and  Manufacturing  Process  Demo 
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MATERIEL  SOLUTION  ANALYSIS  PHASE 
SYSTEMS  ENGINEERING  (SE)  ACTIVITIES 


TECHNOLOGY  DEVELOPMENT  PHASE 
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PRODUCTION  AND  DEPLOYMENT  PHASE 
SE  ACTIVITIES 


Production  Qualification  Testing 


INPUTS  OUTPUTS 


OPERATIONS 

SE 

INPUTS 


AND  SUPPORT  PHASE 
ACTIVITIES 

OUTPUTS 


■Service  Use  Data 
■User  Feedback 
■Failure  Reports 
■Discrepancy  Reports 
■SEP 


Monitor  and  Collect 
All  Service  Use  Data 


'Trades 


Analyze  Data  to 
Determine  Root  Cause 


'Data  for  In-Service  Revievir 
'Input  to  CDD  for  Next  Increment 
'Modifications/Upgrades  to  Fielded 
Systems 
■SEP 


I  Implement  and  Field 
(  Trades  ^ 

Assess  Risk  of 
Improved  System 


Determine  System 

Integrate  and  Test 

Risk/Hazard  Severity 

Corrective  Action 

Develop 

Corrective  Action 


■  Process  Change— 
Hardware/Support 

■  Materiel  Change 
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REQUIREMENTS  (USER  NEEDS) 
ANALYSIS  QUESTIONS 


•  What  are  the  reasons  behind  the  system  development? 

•  What  are  the  customer  expectations?  How  will  they  measure  the 
performance  of  the  system? 

•  Who  are  the  users  and  how  do  they  intend  to  use  the  product? 

•  What  do  the  users  expect  of  the  product? 

•  What  are  their  levels  of  expertise? 

•  With  which  environmental  characteristics  must  the  system  comply? 

•  What  are  existing  and  planned  interfaces? 

•What  functions  will  the  system  perform,  expressed  in  customer 
language? 

•  What  are  the  constraints — hardware,  software,  economic, 
procedural — with  which  the  system  must  comply? 

•What  will  be  the  final  form  of  the  product — model,  prototype,  mass 
production? 


ATTRIBUTES  QF  A  WELL-DEFINED 
REQUIREMENT 

•  Specific,  Ciear,  and  Unambiguous:  Contains  no  vague  terms. 

•  Understandabie:  Stated  with  sufficient  detail  in  everyday 
language. 

•  Concise:  Contains  no  unnecessary  words. 

•  Consistent:  Top-to-bottom  consistency  with  identical  usage  of 
terms  and  conformance  to  standards. 

•  Achievabie:  Reflects  a  need  for  which  a  solution  is  technically 
feasible  at  affordable  costs. 

•Traceabie:  Ultimately  traceable  back  to  a  higher-level  or  stakeholder 
requirement. 

•  Verifiabie:  Expressed  in  such  a  manner  so  that  the  requirement  can 
be  verified  in  an  objective,  preferably  quantitative  manner. 

•  Feasibie:  Can  achieve,  produce,  and  maintain  the  requirement. 
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SYSTEMS  ENGINEERING 
DESIGN  PROCESSES  ILLUSTRATED 


•  Customer  Needs 
Tech  Base 
Prior  Systems  Engineering  Output 

•  Program  Decision  Requirements 
Budget 


Impiied  Req'ts 

Questions  for  Requirers 

'  Technicai  Pianning 
Requirements 
Management 
'  Configuration 
Management 
'  Decision  Anaiysis 
'  Technicai  Assessment 
'  Risk  Management 
'  Interface  Management 
'  Data  Management 

CRUiSE 


Baselines 

1  \ 

Functional 

1 

Product 

(System) 

Aiiocated 

(Detaii) 

(Pert) 

,S| 

System 

Item 

Perf 

|engi^  Icommunicatio^ 


Specifications 


item  Detaii 


X  Materiai 
Process 
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SPECIFICATION  DEVELOPMENT  AND  TECHNICAL  REVIEWS 
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move  to  the  right.  A  PDR  before  MS  B  is  a  statutory  requirement  for  Major  Defense  Acquisition 
Proarams  (MDAPs). 


SYSTEMS  ENGINEERING  TECHNICAL  MANAGEMENT  PROCESSES  KEY  QUESTIONS 
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TECHNICAL  REVIEW  DEFINITIONS 

Alternative  Systems  Review  (ASR):  The  ASR  assesses  the 
preliminary  materiel  solutions  that  have  been  proposed  and  selects 
the  one  or  more  proposed  materiel  solution(s)  that  ultimately  have  the 
best  potential  to  be  developed  into  a  cost-effective,  affordable,  and 
operationally  effective  and  suitable  system  at  an  appropriate  level  of 
risk. 

Critical  Design  Review  (CDR):  The  CDR  establishes  the  initial 
product  baseline.  A  successful  CDR  is  predicated  on  the 
determination  that  the  subsystem  requirements,  subsystem  detailed 
designs,  results  of  peer  reviews,  and  plans  for  test  and  evaluation 
form  a  satisfactory  basis  for  proceeding  into  system  implementation 
and  integration. 

Flight  Readiness  Review  (FRR):  The  FRR  is  a  subset  of  the  Test 
Readiness  Review  and  is  applicable  only  to  aviation  programs.  The 
FRR  assesses  the  readiness  to  initiate  and  conduct  flight  tests  or 
flight  operations. 

Initial  Technical  Review  (ITR):  The  ITR  is  a  multi-disciplined 
technical  review  held  to  ensure  that  a  program’s  technical  baseline 
is  sufficiently  rigorous  to  support  a  valid  cost  estimate  as  well  as 
enable  an  independent  assessment  of  that  estimate. 

In-Service  Review  (ISR):  The  ISR  is  held  to  ensure  that  the  system 
under  review  is  operationally  employed  with  well-understood  and 
managed  risk.  It  provides  an  assessment  of  risk,  readiness,  technical 
status,  and  trends  in  a  measurable  form.  These  assessments  help  to 
substantiate  in-service  support  budget  priorities. 

Preliminary  Design  Review  (PDR):  The  PDR  establishes  the 
allocated  baseline  (hardware,  software,  human/support  systems). 

A  successful  PDR  is  predicated  on  the  determination  that  the 
subsystem  requirements;  subsystem  preliminary  design;  results  of 
peer  reviews;  and  plans  for  development,  testing,  and  evaluation 
form  a  satisfactory  basis  for  proceeding  into  detailed  design  and  test 
procedure  development. 
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Production  Readiness  Review  (PRR):  The  PRR  examines  a 
program  to  determine  if  the  design  is  ready  for  production  and  if 
the  prime  contractor  and  major  subcontractors  have  accomplished 
adequate  production  planning.  The  PRR  determines  if  production  or 
production  preparations  have  unacceptable  risks  that  might  breach 
thresholds  of  schedule,  performance,  cost,  or  other  established 
criteria. 

System  Functional  Review  (SFR):  The  SFR  is  held  to  ensure  that 
the  system’s  functional  baseline  has  a  reasonable  expectation  of 
satisfying  stakeholder  requirements  within  the  currently  allocated 
budget  and  schedule.  The  SFR  assesses  whether  the  system’s 
proposed  functional  definition  is  fully  decomposed  to  its  lower  level, 
and  that  preliminary  design  can  begin. 

System  Requirements  Review  (SRR):  The  SRR  assesses  the 
system  requirements  as  captured  in  the  system  specification  and 
ensures  that  the  system  requirements  are  consistent  with  the 
approved  materiel  solution  (including  its  support  concept)  as  well  as 
available  technologies  resulting  from  any  prototyping  efforts. 

System  Verification  Review  (SVR):  The  SVR  is  held  to  ensure  the 
system  under  review  can  proceed  into  initial  and  full-rate  production 
within  cost  (program  budget),  schedule  (program  schedule), 
risk,  and  other  system  constraints.  The  SVR  assesses  system 
functionality  and  determines  if  it  meets  the  functional  requirements 
as  documented  in  the  functional  baseline. 

Test  Readiness  Review  (TRR):  The  TRR  is  designed  to  ensure  that 
the  subsystem  or  system  under  review  is  ready  to  proceed  into 
formal  test.  The  TRR  assesses  test  objectives,  test  methods  and 
procedures,  scope  of  tests,  and  safety;  and  it  confirms  that  required 
test  resources  have  been  properly  identified  and  coordinated  to 
support  planned  tests. 
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TECHNICAL  REVIEW  BEST  PRACTICES 


Easy  in  principle, 
difficult  in  practice. 


Technical  reviews:  /  z 

^  / 

•  Are  a  fundamental  part  of  the  Systems  Engineerir^g  Tephnical 
Assessment  Process  for  the  program  manager.^ ^  / 

-  Should  be  event-based;  ^  /  / 

-  Objective  entry  and  exit  criteria  need  to  b;e^ define/^  up  front.  See 
the  Defense  Acquisition  Guidebook  (DAG)  for  general  criteria  and 
the  program’s  Systems  Engineering  P^fan  (SEP)' for  specific  criteria; 

-  Are  only  as  good  as  those  who  coipduct  them? 

-  Engagement  of  Technical  Authoi;ify;  / 

-  Chair  independent  of  prograrryfeam;  / 

/  / 

-  Independent  subject-mattej^experts,  det^mined  by  Chair;  and 

-  Involve  all  affected  STAKEHOLDERS.  / 


•  Should  review  status  of  program  develofjment  from  a  technical 
perspective.  ^ 

-  Involve  all  affected  STAKEHOLDERS;  and 


-  Involves  all  technical  products  (e.g.,  specifications,  baselines,  risk 
assessments,  etc.)  relevant  to  the  review. 

•  System-level  reviews  should  occur  after  the  corresponding 
subsystem  level  review! 

NOTE:  Check  out  DAU  Continuous  Learning  Module  CLE003  (Technical  Reviews),  which  includes 
detailed  tailored  checklists  for  all  key  Technical  Reviews. 
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PROGRAM-UNIQUE  SPECIFICATIONS 


Specs 


IVIIL-STD-961 

Standard  Performance  Specification 
Standard  Design  Specification 
Program-Unique  Specifications. 

No  waiver  required  to  use 
STD  PRACTiCE. 


•  Program-unique  specifications  advantages: 

-  Helps  avoid  duplication  and  inconsistencies. 

-  Enables  good  estimates  of  necessary  work  and  resources. 

-  Provides  consistent  communication  among  players  as  people 
rotate. 

-  Can  be  used  to  prepare  test  plans. 

-  Can  be  used  a  long  time  after  the  system  has  been  put  into 
operation. 

-  Serves  as  an  interface  between  customers,  developers,  and 
designers. 

-  Can  act  as  negotiation  and  reference  document  for  engineering 
changes. 


Specification 

Content 

Baseline 

System 

•  Defines  mission/technical  performance 
requirements.  Allocates  requirements  to 
functional  areas.  Defines  interfaces. 

Functional 

(“System”) 

(Hardware  or 
Software)  Item 
Performance 

•  Defines  performance  characteristics  of 
configuration  items  (form,  fit,  function).  Details 
design  requirements  only  to  meet  interfaces. 

“DESIGN-TO.” 

Allocated 

(“Deslgn-to”) 

(Hardware  or 
Software)  Item 
Detail 

•  Includes  “how  to”  and  specific  design 
requirements.  Usually  includes  specific 
processes  and  procedures.  “BUILD-TO.” 

Product 

(“Build-to”) 

Process 

•  Defines  process  performed  during  fabrication. 

Product 

Material 

•  Defines  production  of  raw  materials  or 
semi-fabricated  material  used  in  fabrication. 

Product 
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PERFORMANCE  vs.  DETAIL  SPECIFICATIONS 

Two  generic  categories  of  program-unique  specifications  are  perfor¬ 
mance  specifications  and  detail  specifications. 

•  Performance  Specifications:  States  requirements  in  terms  of 
the  required  results  without  stating  the  method  for  achieving  the 
required  results,  functional  and  performance  (what  and  how  well), 
the  environment  in  which  product(s)  must  operate;  interface  and 
interchangeability  characteristics,  and  criteria  for  verifying  compliance. 

•  Detail  Specifications:  Specifies  requirements  in  terms  of  material  to 
be  used;  how  a  requirement  is  to  be  achieved;  and  how  a  product  is 
to  be  assembled,  integrated,  fabricated,  or  constructed.  Applicable 
to  development  of  contractor  final  design  drawings  as  well  as  items 
being  built,  coded,  purchased,  or  reused. 

•  MIL-STD  961 :  Defense  and  program-unique  specifications  format 
and  content  establishes  the  format  and  content  requirements  for 
defense  specifications  and  program-unique  specifications  prepared 
either  by  DoD  activities  or  by  contractors  for  the  DoD. 


PERFORMANCE 

DETAIL /DESIGN 

Design/Fab. 

Require  desired 
outcomes  or  functions; 
specific  design  to 
contractor 

Specify  exact  parts  and 
components 

Processes 

Few,  if  any 

Specify  exact  processes 

Physical 

Characteristics 

Give  specifics  only  for 
interfaces,  environment, 
or  human  factors 

Specify  more  physical 
characteristics  than 
for  interfaces, 
environment,  etc. 

Interface 

Requirements 

Detailed  interface  data  do 
NOT  solely  make  a  perf. 
spec,  a  detail  spec. 

Detailed  interface  data 

Materials 

Leave  specifics  to 
contractor 

Require  specific  materials 

Test  and  Evaluation 

State  performance 
need;  contractor  picks 
test  procedure 

Prescribed  testing 
process 
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CONFIGURATION  MANAGEMENT 

{Defense  Acquisition  Guidebook,  MIL-HDBK-61A  and 
ANSI/EIA  Standard  649A) 

“A  management  process  for  establishing  and  maintaining  consistency 

of  a  product’s  performance,  functional,  and  physical  attributes  with  its 

requirements,  design,  and  operational  information  throughout  its  life.” 

•  Identify  and  document  the  functional  and  physical  characteristics  of 
configuration  items. 

•  Change  Management:  control  changes  to  configuration  items  and  their 
related  documentation. 

•  Record  (Status  Accounting)  and  report  information  needed  to  manage 
configuration  items  effectively,  including  the  status  of  proposed  changes  and 
implementation  status  of  approved  changes. 

•  Verification  and  Audit  of  configuration  items  to  verify  conformance  to 
specifications,  drawings,  interface  control  documents,  and  other  contract 
requirements. 


CONFiGURATiON  MANAGEMENT  PLANNiNG 
•The  decisions  on 

-  Which  baselines  the  Government  should  eventually  control 

-  The  data  needed 

-  When  that  control  should  be  established 

...  are  strategic  management  decisions  that  involve 

-  Acquisition  strategies — sources,  competition,  etc. 

-  Logistics  support  plans — repair  levels,  data  needs,  open 
systems,  etc. 

-  Technology  insertion — stable  vs.  rapidly  moving  technologies,  etc. 

•  Government  should  control  the  Functional  Baseline  (document  system  level 
requirements) 

•  By  DoD  policy,  at  the  completion  of  the  system  level  Critical  Design  Review 
(CDR),  the  PM  shall  assume  control  of  the  initial  product  baseline  for  all 
Class  1  configuration  changes. 
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INTERFACE  MANAGEMENT 

Will  it  all  work  together? 

The  government  PMO: 

-  Identifies  external  interfaces 

-  Establishes  interface  standards 
(baselines) 

-  Maintains  interface  stability 
The  contractor: 

-  Manages  internal  interfaces 

-  Establishes  interface  requirements 
to  include  internal  and  external 
interfaces 

-  Controls  interfaces  to  ensure 

♦  Accountability 

♦  Timely  dissemination  of  changes 

Since  they  are  not  under  direct  contractor  control, 
the  government  plays  a  big  role  in  managing  external 
interfaces,  leaving  management  and  design  details  of 
internal  system  interfaces  to  the  contractor. 


INTERFACE  CONTROL  CONCEPT 

•  Identifies,  documents,  and  controls  all  functional  and  physical 
characteristics 

•  Interfaces: 

-  What? 

♦  Common  boundary 

♦  Types:  mechanical,  electrical,  operational,  software 

♦  Functional  and  physical  characteristics 

-  Where? 

♦  Within  one  contractor’s  design 

♦  Among  contractor’s  items  and  GFE 

♦  Among  multiple  contractors’  items 

♦  Among  systems 

-  Controlled  by  Interface  Control  Working  Group 

-  Documented  in  Interface  Control  Documents 
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WORK  BREAKDOWN  STRUCTURE  (WBS) 

AWBS: 

•  Is  a  product-oriented  family  tree  composed  of  hardware,  software, 
services,  data,  and  facilities. 

•  Provides  a  consistent  and  visible  framework  for  defense  materiel 
items  and  contracts  within  a  program. 

•  Is  an  organized  method  to  breakdown  the  system  structure  of  a 
product  into  subproducts  at  lower  levels  of  detail. 

•  Can  be  expressed  down  to  any  level  of  interest.  Generally,  the  top 
three  levels  are  sufficient  unless  the  items  identified  are  high  cost  or 
high  risk.  If  so,  then  it  is  important  to  take  the  WBS  to  a  lower  level  of 
definition. 

•  Key  types  of  a  WBS  are  a  Program  WBS  and  a  Contract  WBS 

-  Program  WBS:  encompasses  an  entire  program,  including  the 
Contract  WBS  and  “other  government”  elements  (e.g..  Program 
Office  Operations,  Manpower,  GFE,  government  Testing).  It 
defines  at  a  high  level  what  is  to  be  procured  and  consists  of  at 
least  three  program  levels  with  associated  definitions. 

-  Contract  WBS:  the  complete  WBS  as  extended  to  the  agreed- 
to  contract  reporting  level.  It  defines  the  lower  level  components 
of  what  is  to  be  procured  and  includes  all  the  product  elements 
(hardware,  software,  data,  or  services),  which  are  defined  by 
the  contractor  and  are  their  responsibility. 

•  MIL-HDBK-881 A  (Work  Breakdown  Structures  for  Defense  Materiel 
Items)  should  be  consulted  for  developing  Program  and  Contract 
WBS.  It  provides  details  on  various  types  of  defense  systems  as  well 
as  elements  common  to  all  defense  systems. 

-  MIL-HDBK-881  A  Defense  Systems:  Aircraft  Systems, 
Electronic/Automated  Software  Systems,  Missile  Systems, 
Ordnance  Systems,  Sea  Systems,  Space  Systems,  Surface 
Vehicle  Systems,  and  Unmanned  Air  Vehicle  Systems. 

-  MIL-HDBK-881  A  Common  Elements:  integration, 
assembly,  test,  and  checkout.  Systems  Engineering,  Program 
Management,  training,  data.  System  Test  and  Evaluation, 
peculiar  and  common  support  equipment,  operational  and  site 
activation,  industrial  facilities,  and  initial  spares  and  repair  parts. 
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HOW  TO  CREATE  A 

WORK  BREAKDOWN  STRUCTURE  (WBS) 


Define  Tailor  Create  and  Refine  as 

the  Product  (Supporting  Processes)  Design  Matures 


BASIC  PURPOSES  OF  WBS 
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NOTE:  Oval  shapes  on  periphery 
identify  WBS  purposes 


SOFTWARE  DEVELOPMENT— TECHNICAL  REVIEW  RELATIONSHIPS 
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c 

o 
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NOTE:  SSR— Software  Specification  Review,  The  SSR  is  a  unique  sub-system 
review  heid  to  assess  SW  requirements  prior  to  start  of  design. 


SOFTWARE  DEVELOPMENT— KEY  LIFE  CYCLE  REVIEW  FACTORS 
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CANDIDATE  SOFTWARE  MEASURES  (METRICS) 


Software  Size 
Requirements  Volatility 
Software  Effort/Staffing 
Software  Progress 
Problem/Change  Report  Status 
Rework/Scrap 

Computer  Resource  Utilization 
Milestone  Performance 
Build/Release  Content 
Software  Complexity 
Effect  of  Reuse 
Earned  Value 


Software  measures  should 
be  risk-  or  Issue-driven  and 
are  phase-dependent. 


Check  out  the  handbooks  at  the  “DoD’s  Practical  System  and  Software 
Measures”  site  at  <www.psmsc.com>. 


QUALITY  EVENTS  FOR  SOFTWARE 


Desk 

Checking 


•  Ineffective 

•  Better  than  nothing 

•  Individually  done 


Walk-throughs 


•  May  have  defined  procedures 

•  Team-oriented  review 

•  Results  may  be  recorded 

•  Around  40%  defect  removal 


Spectrum  of 
Quality  Events 
for  Software 


•  Use  specially  trained  teams _ 

•  Formal  process 

Formal  •  Team  attitude  critical  Human-Based 

Inspections  •  Rigid  entry/exit  criteria  Quality  Activities 

•  Basis  for  SW  metrics  I _ _ _ 

•  Genesis  for  process  improvement 

•  Around  70%  defect  removal 


•  Preparation  critical 

Joint  •  Entrance/exit  criteria  key 

Reviews  *  Frequently  abridged 

•  High-level  review 

•  May  not  be  high-leverage, 
SW-quality  event 


Computer- 
_  Based 
Testing 
Activities 


•  Process-driven 

•  Test  and  integration  planning  key 

•  Includes  qualification  testing 

•  Software  item/configuration  Item  oriented 

•  White  vs.  black  box  testing 
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SOFTWARE  MANAGEMENT  BEST  PRACTICES 

•  Adopt  continuous  risk  management 

•  Estimate  cost  and  schedule  empirically 

•  Use  software  metrics  to  help  manage 

•  Track  earned  value 

•  Track  software  defects  against  software  quality  targets 

•  Treat  people  as  the  most  important  resource 

•  Use  life  cycle  configuration  management 

•  Manage  and  trace  requirements 

•  Use  system-based  software  design 

•  Ensure  data  and  database  interoperability 

•  Define  and  control  interfaces 

•  Design  twice,  but  code  once 

•  Carefully  assess  reuse  risks  and  costs 

•  Inspect  requirements  and  design 

•  Manage  testing  as  a  continuous  process 

•  Test  frequently 

•  Use  good  systems  engineering  processes 


SOFTWARE  ACQUISITION  WORST  PRACTICES 

•  Use  schedule  compression  to  justify  new  technology  on  a  time- 
critical  project 

•  Have  the  government  mandate  technological  solutions 

•  Specify  implementation  technology  in  the  RFP 

•  Use  as  many  “silver  bullets”  as  possible 

•  Expect  to  recover  more  than  10%  schedule  slip  without  a  reduction  in 
delivered  functionality 

•  Put  items  out  of  project  control  on  the  critical  path 

•  Plan  on  achieving  more  than  10%  improvement  from  observed  past 
performance 

•  Bury  as  much  of  the  project  complexity  as  possible  in  the  software  as 
opposed  to  the  hardware 

•  Conduct  critical  system  engineering  tasks  without  software  expertise 

•  Believe  that  formal  reviews  alone  will  provide  an  accurate  picture  of 
the  project 

•  Expect  that  the  productivity  of  a  formal  review  is  directly  proportional 
to  the  number  of  attendees  above  five 
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CHAPTER  2 

LEADERSHIP  AND  MANAGERIAL  SKILLS 


•  More  things  that  make  you  go  “Hmmm?...  ” 

“An  authority  is  a  person  who  just  happens  to  know  the  source.” 

“A  conservative  is  a  person  who  believes  nothing  should  be  done 
the  first  time.” 

“Diplomacy  is  the  art  of  hearing  all  parties  arguing  in  a  dispute  and 
nodding  to  all  of  them  without  ever  agreeing  with  any  of  them.” 

“The  meeting  raised  our  confidence  that  the  contractor  can  actually 
accomplish  the  task  and  that  it  will  occur  in  our  lifetime.” 

“This  is  the  earliest  I’ve  been  late.” 

“The  world  would  be  a  much  better  place  if  people  weren’t  allowed 
to  have  children  until  they’ve  proven  they  can  successfully  manage 
a  DoD  program.” 

“Everyone  is  bound  to  bear  patiently  the  results  of  his/her  own 
example.” 

“The  superior  person  is  firm  in  the  right  way,  and  not  merely  firm.” 


MANAGEMENT  AND  LEADERSHIP 


Implement 

Change 

j  MANAGEMENT 
Staff  DoThings 

^Right  ^ 

w  % 

Coordinate 


Organize 


Plan 


Monitor 
&  Control 


MakeThIngs  Happen 


Drive 

Change 


Coach  & 
Mentor 


Build 
Relationships 


%  ^ 

LEADERSHiP 
Do  the 
RightThings 

^  \ 


Set  the 
Direction 


Aiign  the 
People 


Energize 
the  People 


Create  and  Nurture  an  Environment  tor  Success 
Demonstrate  Integrity 


Program 

Managers 

Must 

Balance 

Both 

Roles 
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PROGRAM  MANAGEMENT  “DIAMOND  CARD” 


Designed  by  Al  Moseley,  DAU  SPM 


MANAGEMENT 


•  Logistics 

•  Production 

•  Contracting 

•  Requirements 


•  Vision 

•  EthnicsA/aiues 

•  Teambuiiding 

•  Communication 

•  Leading  Change 

•  Strategic  Thinking 

•  Time  Management 


•  Cost  Estimating 

•  Processes/Toois 

•  Systems  Engineering 

•  Senior  Steering 
Groups 


LEADERSHIP 


•  Organization  Design 

•  Expectation 
Management 

•  Goais 

•  Strategy 

•  Rewards 

•  Partners 


•  Test  &  Evaiuation 
Management 

•  Risk/Funds/Software 
Management 


•  Customers 

•  Environment 

•  Stakehoiders 

•  Teams/Peopie 

•  Reiationship  Buiiding 
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PROGRAM  MANAGEMENT  DIAMOND  CARD  (Continued) 


TECHNOLOGY  READINESS  LEVELS 


System  Test,  Flight 
and  Operations 

cnnpkn 

9  Actual  system  “flight  proven”  through 
successful  mission  operations 

System/Subsystem 

Development 

8  Actual  system  completed  and  “flight 
qualified”  through  test  and  demonstration 

7  System  prototype  demonstration  in  an 
operational  environment 

Technology 

Demonstration 

R 

6  System/subsystem  model  or  prototype 
demonstration  In  a  relevant  environment 

Technology 

Development 

5  Component  and/or  breadboard  validation 
in  relevant  environment 

4  Component  and/or  breadboard  validation 

in  laboratory  environment 

Research  to  Prove 
Feasibility 

A 

3  Analytical  and  experimental  critical  function 
and/or  characteristic  proof-of-concept 

Basic  Technology 
Research 

2  Technology  concept  and/or  application 
formulated 

1  Basic  principles  observed  and  reported 

NOTE:  Technology  Readiness  Levels  (TRL)  enable  consistent,  uniform,  discussions  of  technical  maturity 
across  different  types  of  technologies.  Decision  authorities  wiii  consider  the  recommended  TRLs  when 
assessing  program  risk.  TRLs  are  a  measure  of  technical  maturity.  They  do  not  discuss  the  probability  of 
occurrence  (i.e.,  the  likelihood  of  attaining  required  maturity)  or  the  impact  of  not  achieving  technology 
maturity.  (Defense  Acquisition  Guidebook,  Chapter  10) 
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EMPOWERMENT,  DELEGATION,  AND  COACHING 


EMPOWERMENT 

DELEGATION 

COACHING 

Assigning  an  employee 
or  team  responsibility 
and  authority  to  take 
actions  and  make 
decisions  in  pursuit 
of  the  organization’s 
goals. 

Assigning  an  employee 
(usually  a  subordinate) 
a  specific  task  or  tasks 
to  complete. 

Providing  employees 
with  the  tools, 
knowledge,  and 
opportunities  they 
need  to  develop  their 
potential  and  increase 
their  effectiveness. 

Reasons  for  Empowerment,  Delegation,  and  Coaching 

•  Allows  managers  more  time  for  managerial  and  leadership  roles  (e.g.,  long¬ 
term  planning,  coordinating  ongoing  activities,  monitoring  and  controlling 
activities,  and  providing  feedback  to  employees) 

•  Increases  employee  capability  and  motivation 

•  Enhances  employee  career  growth 

•  Improves  teamwork 

•  Maximizes  limited  resources 

•  Pushes  responsibility  and  accountability  further  down  in  the  organization 

Steps  for  Empowerment,  Delegation,  and  Coaching 

1 .  Select  the  task  or  tasks  to  be  assigned 

2.  Select  the  person  or  team;  evaluate  their  current  capabilities  to  complete 
the  task  or  tasks 

3.  Provide  training  and/or  coaching,  if  necessary,  to  improve  their 
capabilities 

4.  Solicit  input  from  the  person  or  team  regarding  the  task  or  tasks 

5.  Agree  on  the  tasks,  objectives,  responsibiiity,  authority,  and  deadline 

6.  Provide  guidance,  assistance,  and  support,  as  necessary 

7.  Establish  metrics  to  measure  progress 

8.  Monitor  progress 

9.  Provide  feedback 

10.  Identify  lessons  learned 

1 1 .  Evaluate  performance 


NOTE:  Some  people  use  “empowerment”  and  “delegation”  interchangeably,  while  others  see  a  subtle 
distinction,  e.g.,  delegation  often  refers  to  an  individual,  while  empowerment  is  usually  associated 
with  groups  or  teams.  Empowerment  usually  includes  more  authority  and  freedom  related  to  making 
decisions,  and  taking  actions  while  delegation  is  usually  more  bounded. 
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EMPOWERMENT,  DELEGATION,  AND  COACHING 

(Continued) 


Capability 

knowledge,  skill^ 

experience  - 

Direction 

organizational  values, 
vision,  purpose,  goals 

Authority - - 

the  right  to  take  actions 
necessary  to  meet  goals 

Freedom  — ' 

able  to  take  initiative  within 
prescribed  boundaries 


A 

Support 

I 

^ight  tools  and 

-  resources  to  do  job 

/  '  S 

'  /i\  ^ 

Trust 

rt  ''  organizational  values. 

/l\  A 

vision,  purpose,  goals 

/  l\ 

/  1  \ 

— - Resources 

'  ,  \ 

materials,  facilities. 

1  s 

people,  money,  time,  etc. 

1 

K 

-  Information 

/  \ 

/  \ 

access  to  and  sharing  of 
essential  information 

Responsibility  '  ~  Accountability 


or  assignment  to  complete  a  assignment  to  complete  a 

specific  activity  or  activities  specific  activity  or  activities 


Leaders  should  ensure  the  components  shown  above  are  present. 


COACHING  SKILLS 


•  Active  Listening.  Give  your  full  attention.  Focus 
on  the  message,  not  formulating  your  response  to 
it.  Establish  and  maintain  eye  contact,  paraphrase 
key  points,  and  avoid  making  judgments. 

•  Questioning.  Ask  questions  to  promote  discovery 
of  new  knowledge  and  stimulate  thinking.  Use 
open  questions  that  require  some  thought  to 
complete. 

•  Giving  Feedback.  This  is  one  of  the  most 
valuable  yet  least  used  tools  in  communication. 
People  are  often  uncomfortable  giving  feedback 
to  others,  particularly  when  they  believe  it  could 
be  perceived  as  negative.  Offer  factual,  specific, 
but  non-judgmental  (and  unemotional)  feedback. 

•  Sharing.  Share  your  experiences.  Make 
suggestions  on  overcoming  difficulties  or  how  to 
proceed. 
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GENERIC  INTEGRATED  PRODUCT  TEAM  (IPT) 
PROCESS 


Identify  the 
Need  for 
an  IPT 


t  Conduct  Team 
Closeout 
Activities 


Staff  the 
Team 


Conduct 
Team  Startup 
Activities 


Identify  a  Need  for  an  IPT— Determine  whether  the  creation  of  a  team  is  the 
best  method  to  accompiish  the  intended  purpose. 

Staff  the  Team— Determine  what  functionai  discipiines  and  organizations/activities 
need  to  be  represented  and  who  the  team  members  wiii  be. 

Conduct  Team  Startup  Activities— Conduct  activities  to  get  the  team  started, 
such  as  estabiishing  operating  agreements,  assigning  roles  and  responsibilities,  and 
conducting  team  training  sessions.  Activities  also  include  discussing  and  agreeing 
on  the  team’s  intended  purpose  and  developing  shared  goals,  critical  success 
factors,  and  metrics  to  measure  team  progress  toward  goals.  A  common  output  of 
these  activities  is  the  Team  Charter.  (See  page  108.) 

Develop  a  Plan  of  Action— Take  specific  action  steps  or  processes  for  how  the 
team  will  perform.  This  includes  assigning  action  items,  establishing  target  dates, 
determining  what  resources  are  needed,  etc. 

Execute  the  Plan— Perform  the  work  necessary  to  accomplish  the  project  goals 
and  produce  the  team  deliverables. 

Assess  and  Realign— Conduct  periodic  assessments  of  team  performance  and 
use  metrics  to  measure  progress  toward  goals.  Make  adjustments  as  necessary. 

Conduct  Team  Closeout  Activities— Deliver  the  final  product  or  service, 
update  program  documents,  and  compile  lessons  learned. 
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TEAM  CHARTER 

Team  Charter.  A  document  describing  key  aspects  of  why  a  team  is 
established,  what  is  expected  of  it,  and  what  authority  and  responsibil¬ 
ity  it  has.  The  person  or  entity  creating  (i.e.,  “chartering”  or  authoriz¬ 
ing)  the  team  normally  provides  some  general  guidance;  however,  the 
team  may  benefit  considerably  by  developing  the  “meat  and  potatoes” 
of  the  charter,  resulting  in  increased  commitment  of  all  team  members. 
Examples  of  topics  that  may  be  included  in  a  charter  follow: 

•  Purpose.  Describe  why  the  team  exists  and  what  'A  is  intended  to 
accomplish. 

•  Goals/objectives.  List  specific,  measurable  items  the  team  is 
focused  on  achieving  to  help  it  exceed  its  customer’s  expectations. 

•  Critical  success  factors.  List  the  critical  actions  the  team  must 
perform  to  ensure  it  is  successful  in  fulfilling  its  purpose. 

•  End  products/deliverables.  Describe  the  item(s)  the  team  is 
responsible  for  delivering. 

•  Authority  and  accountability.  Describe  what  team  members  are 
allowed/not  allowed  to  do  without  authorization  from  a  higher  level. 
Describe  what  they  are  responsible  for  completing. 

•  Metrics.  List  measures  of  progress  for  critical  success  factors  and 
goals/objectives. 

•  Program  schedule.  List  key  program/team  milestones  and  events. 

•  Team  membership.  List  team  members  and  contact  information. 

•  Roles  and  responsibilities.  List  specific  assignments  for  improving 
team  performance  (e.g.,  timekeeper,  recorder  or  scribe,  scheduler, 
etc.).  Also,  list  specific  tasks  and/or  action  items  the  team  is  assigned 
to  complete. 

•  Resources  required.  Describe  the  funding,  materials,  equipment, 
support,  etc.,  the  team  needs  to  complete  its  mission. 

•  Program  organizational  structure.  Define  where  the  team  fits  within 
the  overall  program  office  structure. 

•  Program  organizational  structure.  Describe  or  depict  where  the 
team  fits  in  the  overall  program  office  structure. 

•  Operating  agreements/ground  rules.  List  agreed-upon  guidelines 
describing  how  team  members  will  interact,  what  processes  they  will 
use,  and  what  they  expect  of  one  another. 

•  Customers,  suppliers,  and  stakeholders.  List  key  individuals, 
teams,  and  organizations  involved  with  the  team’s  output. 
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WORKING  GROUPS 
Team  Development  Wheel 


/  /  Performing 

/  Creative 

/  Trusting 

/  Effective 

Confident 

Milling 

Confusion  \ 

Polite  \ 

Purposeless  \ 

.  Norming 

Storming  / 

\  Cohesion 

Conflict  / 

\  Purpose 

Frustration  / 

Feedback 

Resistance  / 

Relevancy 

Cliques^ 

RECOGNIZE  WHICH  PHASE  OF 
TEAM  DEVELOPMENT  YOU  ARE 
IN  AND  TAKE  POSITIVE  ACTION 
TO  WORK  THROUGH. 


NOTE:  There  can  be  an  additional  phase — “Adjourning” — when  the  team  disbands, 
says  good  bye,  and  reflects  on  lessons  learned.  This  is  a  “celebration”  phase. 

This  diagram  is  based  on  Dr.  Bruce  Tuckman's  1965  study  of  small  groups,  which  identified  the  traditional 
five  phases  experienced  by  project  work  teams. 
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Typical  Working  Groups 

•  Logistics  Support  Management  Team  (LSMT) 

•  Test  and  Evaluation  Working  Group  (TEWG) 

•  Computer  Resources  Working  Group  (CRWG) 

•  Requirements  Interface  Working  Group 

•  Interface  Control  Working  Group  (ICWG) 

•  Technology  Assessment  Working  Group 

•  “Tiger”  Team 

•  Process  Action  Team  (PAT) 

•  Integrated  Product  and  Process  Teams  (IPPTs) 


Management  Tradeoffs  for  Working  Groups 


Advantages 

•  More  ideas  and  solutions 

•  Consensus  positions 

•  Strong  commitments 


Disadvantages 

•  Takes  more  time 

•  Hard  to  terminate 

•  Paralysis  by  analysis 


TEAM  PERFORMANCE  MODEL 


Team  Processes 


Decision  Making 
Resolving  issues 
Communicating 
Planning 
Executing 
Controiiing 


Team  Dynamics 

Diversity 
Confiict 
Comfort  Zones 
Communications 
Focus 

Organizationai  Climate 
Trends 


Team  Foundation 


Customer  Focus 
Leadership 
Vaiues 
Vision 
Purpose 

Goais  and  Objectives 
Criticai  Success  Factors 


Team  Principies 

Awareness 

Roles  and  Responsibilities 
Operating  Agreements 
Team  Accountability 
Empowerment 
Trust 

Five  Cs  - ^ 


Team  Identity 
Self-Assessment 


Communication 

Commitment 

Cooperation 

Contribution 

Caring 
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TEAM  DECISION  MAKING 


Good  team  decision  making  is  a  criticai  eiement  of  team  performance,  it  invoives 
examining  the  decision  context  (e.g.,  current  program  environment,  assumptions,  con¬ 
straints,  pressures,  stakehoider  inputs,  etc.),  determining  who  needs  to  be  involved  in 
the  decision,  verifying  how  much  time  is  available  to  make  the  decision,  and  deciding 
on  the  decision-making  process. 


Generally  Accepted  Team  Decision-Making  Methods 
•  Unilateral.  One  person  makes  the  decision,  usually  the  team  leader. 


Variations: 

—  Directive  or  Authoritative.  The  person  making  the  decision  does  so  primarily 
using  his/her  knowledge,  experience,  and  program  guidelines/constraints,  but 
is  also  influenced  by  his/her  own  reasons  and  motives. 

—  Consultative.  The  person  making  the  decision  may  seek  input  from  other 
team  members,  but  ultimately,  he/she  still  makes  the  decision  on  his/her  own. 

•  Majority.  Each  team  member  votes,  and  the  majority  decides  the  course  of  action. 

•  Consensus.  Team  members  may  not  completely  agree  with  the  most  preferred 
approach,  but  they  have  the  opportunity  to  express  their  point  of  view,  understand 
the  logic  behind  the  decision,  and  support  it.  Consensus  is  generally  the  preferred 
decision-making  method  for  most  team  issues,  especially  when  the  commitment  of 
all  team  members  is  important. 


Guidelines  for  Achieving  Consensus: 


—Don’t  try  to  force  consensus.  Listen  to  other  positions  and 
reactions  before  expressing  your  own  point. 


—No  winners  or  losers.  Don’t  assume  that  someone  must  “win” 
and  someone  must  “lose”  if  the  discussion  reaches  a  stalemate. 

—Don’t  avoid  conflict.  Don’t  change  your  mind  simply  to  reach 
agreement  and  maintain  harmony. 

—Avoid  majority  votes,  compromises,  or  horse  trading  to  reach  an 
agreement. 

—It’s  OK  to  disagree.  Differences  of  opinion  are  natural  and 
expected. 


NOTE:  Groupthink.  A  phenomenon— to  be  avoided— where  team  members  become  so  concerned  about 
preventing  disagreement  or  conflict  that  they  abandon  critical  thinking  to  simply  go  along  with  whatever 
consensus  seems  to  be  emerging. 
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EFFECTIVE  MEETINGS 

Prior  to  the  Meeting 

•  Determine  and  clarify  the  purpose  for  the 
meeting 

•  Determine  expected  meeting  outcomes 

•  Identify  meeting  attendees 

—  Subject-matter  experts 

—  Key  decision  makers 

—  People  directly  affected  by  potential 
decisions/outcomes 

•  Determine  meeting  format 

—  Face-to-face,  virtual  teleconference, 
teleconference,  Web  tool 

•  Determine  date/time/location 

•  Develop  and  distribute  meeting  agenda  (at  least  24  hours  prior) 

—  Specific  topics,  presenter,  estimated  time,  desired  outcome 

•  Meeting  logistics 

—  Room  setup,  IT  support  needed 

During  the  Meeting 

•  Opening 

—  Start  on  time 

—  Review  agenda 

—  Set  or  review  ground  rules 

—  Clarify  roles 

•  Conducting 

—  Address  one  item  at  a  time 

—  Facilitate  discussions 

—  Encourage  open  communication  and  information  sharing 

—  Maintain  focus  and  pace 

—  Specify  topics,  presenter,  amount  of  time  devoted  to  item 

•  Closing 

—  Summarize  agreements  and  decisions 

—  Review  action  items 

—  Ask  for  agenda  items  for  the  next  meeting 

—  Set  the  date/time  of  the  next  meeting 

After  the  Meeting 

Review  and  publish  minutes 
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DECISION  BRIEFING 


Elements  of  a  Decision  Briefing 

•  Outline — Agenda 

•  Purpose  of  Briefing/lssue(s) 

•  Background 

•  Assumptions 

•  Alternatives  Identified 

•  Evaluation  Criteria/Process 

•  Analysis  of  Identified  Alternatives 

•  Recommended  Alternative 

•  Rationale  for  Recommendation 

•  Recommended  Implementation  Plan 

•  Key  Risks  for  Recommended  Implementation  Plan 

What  to  Expect  from  the  Person/People  Receiving  the  Briefing 

•  Challenges  to  assumptions,  definitions,  methodology 

•  Questions  concerning  compliance  with  or 
changes  to  policy 

•  Sensitivity  of  the  issue  and/or 
recommended  alternative  to  change 

•  Questions  or  challenges  to  analysis, 
tradeoffs,  rationale  for  recommendations,  and  implementation  plan 

•  Questions  concerning  risks  for  the  recommended  implementation 
plan 

NOTE:  Questions  may  be  open-ended  or  closed  (e.g.,  yes/no  answers). 
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COMMUNICATIONS 


Messages  pass  through  filters;  first  through  the  filter  of  the  person 
sending  the  message  and  then  through  the  filter  of  the  receiver.  Filters 
sometimes  act  to  enhance  the  message,  and  at  other  times,  they  can 
be  barriers.  Filters  consist  of  factors  such  as  personality,  tone  of  voice, 
body  language,  facial  expressions,  accents,  perceptions,  attitudes, 
emotions,  knowledge,  functional  background,  the  medium  of  com¬ 
munication  used  (verbal,  written,  e-mail,  etc.),  and  much  more.  Each 
person’s  filter  is  different,  sometimes  resulting  in  the  receiver  interpret¬ 
ing  the  message  differently  than  the  sender  intended. 

One  of  the  most  important  communications  skills  (and  often  a  barrier 
to  effective  communications)  is  listening.  Learning  to  “actively  listen” 
can  increase  communications  effectiveness  significantly 

Active  listening  involves: 

•  Establishing  and  maintaining  eye  contact. 

•  Focusing  on  what  is  being  communicated. 

•  Not  making  judgments  about  the  sender’s 
information. 

•  Not  formulating  your  reply  before  the  sender  has 
finished  sending  his/her  message. 

•  Paraphrasing  key  points  the  sender  makes  (when  the  sender 
pauses — don’t  interrupt  to  paraphrase  what’s  being  communicated). 

Effective  program  management  requires 
that  the  right  people  get  the  right  informa¬ 
tion  at  the  right  time.  Program  communica¬ 
tions  must  take  place  vertically  (up  and 
down),  horizontally,  and  externally. 


Program  Office 


¥  4 
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Communications  Pian 

One  way  to  ensure  the  right  people  get  the  right  information  at  the  right  times  is  to  develop  a 
program  (and/or  team)  communications  plan.  The  plan  may  include: 

•  Key  entities  (program  management  leadership,  IPTs,  customer,  contractor(s),  and  key 
stakeholders) 

•  What  information  they  should  provide 

•  What  information  they  should  receive 

•  How  it  is  provided/received 

•  Format,  frequency/interval,  and  other  factors  considered  important  for 
the  particular  program/situation 

•  Types  of  meetings,  such  as  regular  status  meetings  and  program 
management  reviews 

•  Reports  (e.g.,  status  reports,  cost/schedule  performance  reports,  action  item  lists) 

•  Issues  and  the  policy  for  elevating  them  to  higher  levels 

•  Other  forms  of  communication  and  how  and  by  whom  they  are  used 

Interpersonal  Negotiation  Techniques 
Purpose:  Resolving  conflicts 
Objective:  Seek  to  satisfy  both  parties’  interests 

Methodology: 

—  Acknowledge  the  conflict  and  its  effect  on  performance. 

—  Separate  people  and  emotions  from  the  issue. 

—  Present  issues  in  terms  of  the  underlying  interests  or 
requirements,  i.e.,  the  most  important  aspects  of  what  you  need 
to  achieve. 

—  LISTEN  to  the  other  party’s  interests/requirements;  be  able 
to  restate  their  interests  to  their  satisfaction  (indicating  you 
understand  what  interests  they  are  trying  to  achieve). 

—  Agree  on  what  the  issue  is. 

—  Look  for  common  goals  and  common  interests. 

—  Identify  as  many  possible  alternatives  to  resolve  the  issue  and  satisfy  the  interests  of 
both  parties. 

—  Resist  the  urge  to  compromise  (“meet  in  the  middle’’).  Instead,  look  at  the  issue  from 
different  perspectives— challenge  assumptions  and  constraints. 

—  Agree  on  the  alternative  that  best  meets  both  parties'  interests. 

—  Obtain  the  commitment  of  all  members  of  both  parties  on  what  will  be  done  to  implement 
the  solution. 
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COUNSELING 


f 

DIRECTIVE 

•  Give  advice 

•  Evaluate 

•  Motivate 

•  Explain 

•  Reassure 

Advantages 

•  Effective  with  inexperi¬ 
enced  personnel 

•  Quick 

•  Take  charge  attitude 

Disadvantages 

•  Perceived  insulting 

•  Does  not  support  delegation 

•  Manager  keeps  responsibility 


NONDIRECTIVE 

•  Don’t  display  authority 

•  Listen  carefully 

•  Don’t  advise 

•  Facts  only;  no  opinions 

•  Employee  find  solution 

Advantages 

•  Develops  commitment 

•  Good  training 

•  Employee  responsible 

•  Supports  delegation 

Disadvantages 

•  Takes  time 

•  Skill/patience  required 

•  Ineffective  with  inexperi 
enced  personnel 


COUNSELING  PROCESS 

1.  Set  up  interview — private,  confidential,  and 
unhurried 

2.  Encourage  discussion — open  questions, 
active  listening 

3.  Help  employee  think  it  through — deal  with 
facts,  no  opinions  or  own  views 

4.  Let  employee  find  the  solution — his/her  solution 
to  the  problem 
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TIME  MANAGEMENT 

1 .  List  all  the  tasks  you  have  to  complete. 

2.  Prioritize  the  tasks  based  on  urgency  and  importance  of  completion 
using  the  format  shown  below. 

3.  Do  Priority  1  tasks  first.  If  possible  delegate  some  of  them. 

4.  The  key  to  effective  time  management  is  to  schedule  time  to  work  on 
small  pieces  of  Priority  2  tasks. 

—  If  not  completed  early,  they  will  eventually  become  Priority  1  tasks. 

5.  Reassign  or  delegate  Priority  3  tasks  if  possible. 

—  A  common  tendency  is  focusing  on  Priority  3  tasks  (because  of  their 
urgency)  instead  of  Priority  2  tasks  (because  of  their  importance). 

6.  Priority  4  tasks  are  time  wasters/busy  work  and  should  be  avoided. 


Priority  1  Important 

Urgent 

Priority  2  Important 

Not  Urgent 

Priority  3  Urgent 

Not  Important 

Priority  4  Not  Urgent 

Not  Important 

Common  Time  Robbers  Avoidance  Techniques 


•  Incoming  telephone  calls 


•  Outgoing  telephone  calls 


•  Unscheduled  visitors 


•  Improper  delegation 


•  Poorly  conducted  meetings 


✓  Screen  for  importance 

✓  Allow  voice  mail  to  pick  up  the  call 

✓  Limit  length  of  calls  (e.g.,  2  min.) 

✓  Do  as  many  at  one  time  as  possible 

✓  Itemize  topics  before  calling 

✓  Stick  to  the  topic;  don't  socialize 

✓  Screen  for  importance 

✓  Do  not  invite  visitor  into  your  office 

✓  Remain  standing 

✓  Schedule  a  time  for  visitor  to  return 

✓  Re-delegate 

✓  Make  a  record  of  delegated  tasks 

✓  Assign  deadlines 

✓  Have  a  prepublished  agenda 

✓  Stay  focused  on  subject 

✓  Use  a  time  keeper/gate  keeper 
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MANAGEMENT  TOOLS  AND  TECHNIQUES 

1 .  Activity-Based  Management  (ABM)  uses  detailed  economic 
analyses  of  important  business  activities  to  improve  strategic 
and  operational  decisions.  ABM  increases  the  accuracy  of 
cost  information  by  more  precisely  linking  overhead  and  other 
indirect  costs  to  products  or  customer  segments.  Traditional 
accounting  systems  distribute  indirect  costs  using  bases  such 
as  direct  labor  hours,  machine  hours,  or  materiel  dollars.  ABM 
tracks  overhead  and  other  indirect  costs  by  activity,  which  can 
then  be  traced  to  products  or  customers. 

2.  Balanced  Scorecard  defines  what  management  means 
by  “performance”  and  measures  whether  management  is 
achieving  desired  results.  The  Balanced  Scorecard  translates 
mission  and  vision  statements  into  a  comprehensive  set  of 
objectives  and  performance  measures  that  can  be  quantified 
and  appraised.  These  measures  typically  include:  financial, 
customer  value,  internal  business  process,  learning  and  growth, 
and  employee  performance. 

3.  Cycle  Time  Reduction  decreases  the  time  it  takes  a  company 
or  program  to  perform  key  activities  throughout  its  value  chain. 
Cycle  Time  Reduction  uses  analytic  techniques  to  minimize 
waiting  time,  eliminate  activities  that  do  not  add  value,  increase 
parallel  processes,  and  speed  up  decision  processes  within  an 
organization.  Time-based  strategies  often  emphasize  flexible 
manufacturing,  rapid  response,  and  innovation  in  order  to 
attract  the  most  profitable  customers. 

4.  Groupware  refers  to  a  broad  range  of  technologies  that  allow 
people  in  organizations  to  work  together  through  computer 
networks.  These  products  range  from  sophisticated  electronic 
mail  packages  to  applications  that  link  offices  and  employees. 
Organizations  use  such  technology-aided  communications 

to  better  inform  strategic  and  financial  decisions  and  to  more 
effectively  and  economically  bring  together  working  groups. 
(DAU  has  a  Groupware  capability  in  its  Management  Decision 
Center,  which  is  used  for  management  decision  making  by 
offices  and  agencies  throughout  DoD.) 
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5.  Outsourcing  occurs  when  a  company  or  government  agency 
uses  third  parties  to  perform  non-core  business  activities. 
Contracting  third  parties  enables  a  company  or  agency  to  focus 
its  efforts  on  its  core  competencies.  Many  companies  find  that 
outsourcing  reduces  cost  and  improves  performance  of  the 
activity.  Third  parties  that  specialize  in  an  activity  are  likely  to 
be  lower  cost  and  more  effective,  given  their  scale.  Through 
outsourcing,  a  company  or  agency  can  access  the  state  of  the 
art  in  all  of  its  business  activities  without  having  to  master  each 
one  internally. 

6.  Business  Process  Reengineering  involves  the  fundamental 
redesign  of  core  business  processes  to  achieve  significant 
improvements  in  productivity,  cycle  times,  and  quality.  In 
Business  Process  Reengineering,  companies  start  with  a 
blank  sheet  of  paper  and  rethink  existing  processes  to  deliver 
more  value  to  the  customer.  They  typically  adopt  a  new  value 
system  that  places  increased  emphasis  on  customer  needs. 
Companies  and/or  government  agencies  reduce  organizational 
layers  and  eliminate  unproductive  activities  in  two  key  areas: 
First,  they  redesign  functional  organizations  into  cross¬ 
functional  teams.  Second,  they  use  technology  to  improve  data 
dissemination  and  decision  making. 

7.  Strategic  Planning  is  a  comprehensive  process  for 
determining  what  a  commercial  business  or  government 
agency  should  become  and  how  it  can  best  achieve  that  goal. 

It  appraises  the  full  potential  of  a  business  and  explicitly  links 
the  business  objectives  to  the  actions  and  resources  required 
to  achieve  them.  Strategic  Planning  offers  a  systematic  process 
to  ask  and  answer  the  most  critical  questions  confronting  a 
management  team — especially  large,  irrevocable  resource 
commitment  questions. 
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CONTINUOUS  PROCESS  IMPROVEMENT/ 

LEAN  SIX  SIGMA  (CPI/LSS) 

(DoD  Directive  5010.42, 15  May  2008) 

•  “Lean”  and  “Six  Sigma”  are  actually  two  distinct  process  improvement  ideas 
often  merged  together  forming  “Lean  Six  Sigma.” 

•  “Sigma”  is  the  term  used  for  standard  deviation — a  statistical  measure  of 
variation.  Variation  can  be  decreased  via  Standard  Operating  Procedures 
(SOPs)  and  training. 

•  “Lean”  references  a  process  that  can  be  shortened  by  eiiminating  non-vaiue- 
added  steps. 

•  Operating  at  a  “Six  Sigma”  levei  of  performance  means  the  process 
theoretically  produces  a  99.99966%  defect-free  yield  or  3.4  Defects  Per 
Million  Opportunities  (DPMO). 


*  Five-Step  Process:  Define,  Measure,  Analyze,  Improve,  Control  (DMAIC) 


Decide  Which  Opportunities  Require  Lean  Six  Sigma  and 
Which  Require  Just  Lean  or  Six  Sigma 


If... 

Then  ... 

•  Slow,  wasteful  business  processes 
are  the  problem 

•  Little  or  no  historical  process  data 

•  GOAL:  Process  speed 

•  Focus  on  Lean 

—  Elimination  of  “non-value  added” 
activities 

—  Workflow  simplification 

•  Streamlined,  highly  efficient 
business  processes 

•  Process  variability  identified  as 
a  problem  based  on  statistically 
significant  historical  data 

•  GOAL:  Process  consistency 

•  Focus  on  Six  Sigma 

—  Elimination  of  variation  factors 
and  lack  of  control 

—  Data-driven  management 

•  Slow,  wasteful  business  processes 
combined  with  low  performance  or 
quality  variability 

•  GOAL:  Speed  and  consistency 

•  Focus  on  integrated  Lean  and 

Six  Sigma 

—  Methodology  as  a  total  perfor¬ 
mance  solution 

—  Total  process  view  with  embed¬ 
ded  measurement  and  assess¬ 
ment  capabilities  leading  to  posi¬ 
tive  performance  management 
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CHAPTER  3 

PROBLEM-SOLVING  TOOLS 

BRAINSTORMING 

PURPOSE:  To  stimulate  the  free  flow  of  ideas  in  a  short  amount  of  time 
without  being  analyzed  or  judged  until  the  brainstorming  is  complete. 

METHOD:  There  are  three  primary  types  of  brainstorming:  structured, 
unstructured,  and  silent. 


•  Structured:  Participants  take  turns  offering  ideas;  if  someone  doesn’t 
have  an  idea  when  their  turn  comes,  they  can  pass. 

—  Advantage:  Each  person  has  an  equal  chance  to  participate. 

—  Disadvantages:  Lacks  spontaneity;  participants  may  get 
distracted  by  other  ideas  and  forget  theirs  when  their  turn 
comes,  atmosphere  is  more  rigid. 

•  Unstructured:  Participants  offer  ideas  as  they  think  of  them. 

—  Advantage:  Participants  can  build  on  each  others’  ideas; 
atmosphere  is  more  relaxed. 

—  Disadvantage:  Less  assertive  and/or  lower  ranking  participants 
may  feel  intimidated  and  not  contribute. 

•  Silent:  Participants  write  ideas  individually  on  paper  or  Post-it™ 
notes.  This  is  particularly  useful  when  you  have  participants  who  just 
can’t  avoid  discussing  the  ideas  as  they  are  offered. 

—  Advantage:  Prevents  discussion  of  ideas  during  the  idea 
generation  phase. 

—  Disadvantages:  May  lose  the  opportunity  to  build  on  others’ 
ideas  unless  a  structured  or  unstructured  session  is  held  after 
the  silent  inputs  are  collected  and  displayed. 

The  brainstorming  session  ends  when  no  more  ideas  are  offered. 
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GROUND  RULES: 

Don’t  discuss  ideas  as  they  are  offered.  In  particular,  don’t 
analyze,  evaluate,  criticize,  or  judge.  Discussion  can  be  held 
after  the  brainstorming  session  ends. 

There  are  no  outrageous  ideas.  There  is  plenty  of  time  during 
the  discussion  after  the  brainstorming  session  to  toss  out  ideas 
that  won’t  work.  Even  if  idea  is  totally  outrageous  and  obviously 
won’t  work,  it  may  spark  another  idea  that  is  usable. 

Don’t  quit  when  the  ideas  first  stop  fiowing;  try  to  get  partici¬ 
pants  to  come  up  with  at  least  2-3  more  ideas. 

Strive  for  quantity,  not  quaiity.  The  more  ideas  you  generate, 
the  better  the  opportunity  to  find  the  best  possible  solution. 

Combine  and  rearrange  ideas;  additions,  revisions,  and  combi¬ 
nations  may  create  even  better  ideas. 

Record  ideas  exactiy  as  offered,  don’t  edit  or  paraphrase. 

QUESTIONS  TO  STIMULATE  YOUR  THINKING: 

1 .  Can  we  use  this  idea  somewhere  else?  As  is?  With  changes? 

2.  If  we  change  it,  is  there  anything  else  like  it?  Any  related 
issues? 

3.  Can  we  modify  or  rearrange:  the  meaning,  quantity,  color,  size, 
shape,  form,  layout,  motion,  sound,  appearance,  etc.? 

4.  Can  we  maximize  or  magnify  it  to  make  it  stronger,  larger, 
newer,  more  of  it? 

5.  Can  we  minimize  or  reduce  it  to  make  it  smaller,  lighter,  less  of 
it? 

6.  Can  we  substitute?  Who?  What?  When?  Where?  How? 

7.  Can  we  reverse  it?  Opposite?  Backwards?  Upside  down? 
Inside  out? 

8.  What  assumptions  or  constraints  are  we  considering?  Are  they 
valid?  What  if  we  threw  them  out? 

9.  What  if  you  could  do  anything  you  can  imagine? 
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CAUSE-AND-EFFECT  DIAGRAM 

(“FISHBONE”  OR  ISHIKAWA  DIAGRAM) 

PURPOSE:  To  help  analyze  a  problem  in  increasing  detail  to  identify 

all  of  its  causes,  leading  to  discovery  of  its  root  cause(s).  The  Cause- 

and-Effect  Diagram  graphically  depicts  the  relationship  between  a 

problem  and  its  causes. 

METHOD: 

•  Use  brainstorming  to  generate  the  potential  or  known  causes  for  the 
problem  (or  effect)  being  studied. 

•  Begin  constructing  the  “fishbone”  diagram  by  placing  the  problem 
statement  on  the  right  side  of  the  chart  (head  of  the  “fishbone”). 

•  Draw  an  arrow  from  left  to  right  ending  at  the  problem  statement  (the 
backbone  of  the  “fishbone”). 

•  Place  the  major  cause  categories  (If  known)  as  the  major  “bones’ 
of  the  fishbone,  as  shown  in  the  example  below  (in  the  example: 
people,  product,  process,  equipment). 

•  If  the  major  causes  are  not  known,  after  brainstorming  all  the 
causes,  sort  them  into  similar  groups  using  the  Affinity  Diagram.  The 
titles  of  the  groups  become  the  major  cause  categories. 

•  Add  the  brainstormed  causes  as  the  smaller  bones  in  the  diagram 
as  shown  in  the  example  below  (e.g.,  inadequate  training,  poor 
teamwork,  etc.).  Causes  can  be  added  to  the  major  categories 
after  all  the  causes  have  been  generated  via  brainstorming 
(recommended),  or  added  as  they  are  generated. 

•  To  spark  additional  brainstorming  of  causes,  ask  for  each  of  the 
“small  bone”  causes:  “What  causes  this  to  happen?” 


EXAMPLE: 
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FORCE  FIELD  ANALYSIS 

PURPOSE:  To  identify  the  factors  or  forces  that  either  support  or  work 

against  a  desired  outcome. 

METHOD: 

•  Draw  a  “T”  shape  as  shown  below. 

•  Brainstorm  the  forces  that  will  assist  you  in  achieving  the  desired 
outcome.  List  them  on  the  left  side  of  the  vertical  line. 

•  Brainstorm  the  forces  that  may  prevent  or  restrain  you  from  reaching 
your  outcome.  List  them  on  the  right  side  of  the  line. 

•  (Optional)  Prioritize  the  driving  forces  (left  side)  and/or  the  restraining 
forces  (right  side). 

•  Look  for  opportunities  to  take  advantage  of  or  strengthen  driving 
forces. 

•  Identify  restraining  forces  that  you  might  be  able  to  eliminate  (or 
reduce  the  “force”  or  impact). 

•  It  is  often  more  helpful  to  eliminate  restraining  forces  than  attempting 
to  strengthen  driving  forces.  In  most  cases,  the  driving  forces  will 
remain  present  and  continue  to  help  you  even  if  you  do  nothing  to 
strengthen  them;  whereas  eliminating  restraining  forces  can  have 
significant  benefits  in  achieving  your  objective/outcome. 

•  In  a  “pound-for-pound”  or  “best  bang  for  the  buck”  fashion,  the  force 
field  analysis  is  one  of  the  most  powerful  tools  in  terms  of  the  effort 
required  to  generate  it  and  the  potential  benefits  derived  from  it. 

•  Restraining  forces  can  also  be  identified  as  potential  risks,  and 
entered  into  the  risk  management  tracking  system. 

EXAMPLE: 

OBJECTIVE:  Successfully  Complete  Preliminary  Design 


+  Driving  Forces - ►  -  Restraining  Forces 


pypArlftncAri  tppimlpaHpr  ^ 
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HISTOGRAM 

PURPOSE:  To  graphically  depict  the  frequency  distribution  of  data  items 

using  a  vertical  bar  chart  (columns)  format. 

METHOD: 

•  Collect  data  on  a  particular  variable. 

•  Generate  a  frequency  table  listing  all  the  data  points. 

•  Count  the  number  of  data  points. 

•  Determine  the  range  of  values  for  the  data  (maximum  value  minus 
the  minimum  value). 

•  Determine  the  number  of  bars  to  depict  in  the  chart.  One  common 
method  is  to  use  the  square  root  of  the  number  of  data  points;  e.g., 

1 00  data  points  =  1 0  bars;  225  data  points  =  1 5  bars,  etc. 

•  Calculate  the  intervals  represented  by  each  of  the  bars.  The  simplest 
method  is  to  divide  the  range  of  values  by  the  number  of  bars  (from 
previous  step). 

•  Determine  the  frequency  of  data  points  within  each  interval  of  values. 

•  Create  a  vertical  bar  chart  with  a  vertical  bar  (column)  for  each  of  the 
variable  values  or  range  of  values  you  measured  on  the  horizontal 
axis.  The  height  of  the  bar  will  equal  the  frequency  (on  the  vertical 
axis)  for  each  of  the  values/ranges. 

EXAMPLE: 

•  In  the  example  below,  the  sample  size  is  220  data  points  (N=220). 

•  The  square  root  of  220  is  between  14  and  15,  so  either  will  work  for 
the  number  of  bars  in  the  chart  (14  bars  are  used  in  the  example). 

•  The  range  of  values  is  350  hrs  (1350  hrs  minus  1000  hrs).  Dividing 
the  range  (350)  by  the  number  of  bars  (14)  results  in  intervals  of  25 
hrs. 
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SCATTER  DIAGRAM 

PURPOSE:  To  graphically  depict  the  changes  in  two  variables  to  de¬ 
termine  if  there  is  a  relationship  between  them. 

METHOD: 

•  Collect  paired  data  samples  for  the  two  variables. 

•  Place  measures  for  the  independent  variable  (the  hypothesized 
cause)  on  the  horizontal  axis,  and  measures  for  the  dependent 
variable  (the  hypothesized  effect)  on  the  vertical  axis. 

•  Plot  the  data  on  the  chart 

•  Analyze  the  data  to  determine  if  there  is  a  statistical  relationship 
between  the  two  variables. 


Positive  Correiation  Negative  Correlation  No  Correlation 

For  an  in  creasein  “X”  For  an  in  creasein  “X”  For  an  in  creasein  “X” 

there  is  a  corresponding  there  is  a  corresponding  there  is  no  corresponding 

increase  in  “Y”  decrease  in  “Y”  reaction  in  “Y” 


EXAMPLE: 
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SURVEYS 

Surveys  are  used  to  collect  data  from  a  variable  number  of  items  or  people 
for  a  comparative  study.  They  are  used  when  a  new  project  is  planned  to 
prove  the  need  and  the  demand  of  the  customer. 

Surveys  can  be  used  anywhere  in  the  organization  to  find  out  specific 
information  that  is  necessary  to  make  improvements  in  a  process. 

Surveys: 

•  Are  an  inexpensive  way  to  test  a  system  or  product; 

•  Can  be  used  with  a  large  number  of  people  or  a  small  group; 

•  Can  give  you  an  overall  view,  determined  by  the  questions  you  ask; 

•  Show  if  an  organization  is  meeting  its  quality  goals;  and 

•  Help  identify  satisfied  and  dissatisfied  customers  or  employees. 

Survey  Process 

1 .  Determine  the  group  to  be  studied. 

2.  Determine  what  questions  will  be  asked. 


BRAINSTORMING  SURVEY 


Note:  Train  your  data  collectors  thoroughly.  Everyone  must  know  how  to  ask  the 
questions,  whom  to  approach,  and  how  to  approach  them. 

3.  Compile  your  results  in  chart  form  using  a  Pareto  Chart  (see  page 
130),  histogram,  and  other  tools  that  will  give  you  clarification. 

4.  Use  the  compounded  data  to  form  a  base  for  improvement. 

5.  Continue  to  take  data  to  monitor  improvements  and  make  sure  the 
improvements  you  have  made  are  working. 

Caution! 

•  Data  must  be  collected  honestly  and  consistently. 

•  An  untrained  collector  can  skew  the  data  to  reflect  personal  biases. 

•  A  poor,  inconsistent  survey  will  give  you  invalid  data. 

•  Make  sure  there  is  enough  time  allowed  for  the  collecting  process. 
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AFFINITY  DIAGRAM 

PURPOSE:  To  help  a  team  generate  a  large  number  of  ideas  or  issues 

and  organize  them  into  categories  for  further  analysis,  evaluation,  deci¬ 
sion,  or  action. 

METHOD: 

•  Agree  on  what  the  problem,  issue,  question,  or  desired  outcome  is. 

•  Brainstorm  as  many  ideas  about  the  problem,  issue,  question,  or 
desired  outcome  as  you  can. 

•  Without  discussion,  sort  the  ideas  into  related  groups. 

•  If  two  people  can’t  agree  on  which  category  it  best  fits,  consider 
duplicating  it  and  including  it  under  both. 

•  For  each  group,  identify  a  name  that  summarizes  the  topics  listed  for 
them. 

Tip:  Use  Post-it™  type  notes  to  record  the  ideas  on,  which  allows  you 
to  easily  move  the  ideas  from  one  category  to  another. 

EXAMPLE: 

What  are  the  Attributes  of  an  Effective  Leader? 


integrity 

Coaching/Mentoring 

Honesty 

Trustworthy 

Does  the  right  things 
Loyal 

Courageous 

Sets  the  example 

Provides  career 
counseiing 

Supports  training 
Provides  professionai 
growth  opportunities 

Visionary 

Change  Agent 

Has  a  big  picture  view 

Drives  change 

Cieariy  stated  goais 

Innovative 

Decisive 

Knowledgeable 

Makes  decisions 

Strong  technical 
expertise 

involves  others  in 
decisions 

Motivation 


Inspires 

Focused  on  people 

Trusting 

Caring 

Sets  high  standards 

Empowers 

Supportive 


Communication 


Good  iistener 
Effective  speaker 
Encourages  open 
communication 


Note:  Attributes  shown  above  are  for  illustration  only,  and  not  meant  to  portray  actual  answers  to  the 
question. 
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PAIRWISE  RANKING 

PURPOSE:  To  provide  a  structured  method  for  ranking  small  lists  of 

items  in  priority  order. 

METHOD:  1 

•  Construct  a  pairwise  matrix. 

—  Each  of  the  squares  in  the  matrix  at 
right  represents  the  pairing  of  two 
items  (where  the  numbers  intersect). 

—  In  this  example,  the  list  includes 
five  items;  the  top  square  (shaded) 
represents  the  pairing  of  item  1  with 
item  2. 

•  Rank  each  pair  of  items. 

—  For  each  pair  of  items,  the  team 
should  reach  a  consensus  on  which  of  the  two  items  is 
preferred  over  the  other. 

—  As  the  team  completes  each  of  the  comparisons,  the  number  of 
the  preferred  item  is  recorded  in  that  square,  until  the  matrix  is 
completely  filled  in. 


1  1  1 


2 

2 

2 

2 

2 

2 

2 

2 

2 

3 

3 

3 

1 

3 

3 

1 

2 

3 

4 

4 

4 

4 

4 

1 

2 

3 

4 

5 

5 

5 

5 

5 

5 

5 

1  and  2  are  compared;  1  and  3  are  compared;  ...and  finally,  4  and  5  are 
2  is  preferred  1  is  preferred  compared;  5  is  preferred 


•  Count  the  number  of  times  each  item  appears  in  the  matrix. 

—  Using  the  filled-in  matrix  (on  the  far  right  above),  count  how 
many  times  each  item  is  listed  in  the  matrix,  and  record  the 
totals  in  the  ranking  matrix  (at  right). 


Alternative 

1 

2 

3 

4 

5 

Count 

2 

3 

1 

0 

4 

Rank 
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•  Rank  all  of  the  items. 

—  Rank  the  items  based  on  how  many  times  they  appear  in  the 
matrix. 

—  To  break  a  tie  between  two  items  appearing  the  same  number 
of  times,  look  at  the  square  in  the  matrix  where  the  two  were 
compared;  the  item  appearing  in  that  box  receives  the  higher 
ranking. 


Items  5  appear  four  times 
in  the  matrix,  so  it  ranks 
1st  (see  shaded  squares  in 
the  matrix  at  right);  item  2 
appears  three  times,  which 
ranks  2nd,  etc. 


Alternative 

1 

2 

3 

4 

5 

Count 

2 

3 

1 

0 

4 

Rank 

3rd 

2nd 

4th 

5th 

1st 

EXAMPLE: 


A  program  team  was  asked  to  recommend  a  site  for  testing  a  unique  portion  of 
a  system.  A  feasibility  study  produced  a  iist  of  six  possibie  locations.  The  team 
then  used  Pairwise  Ranking  to  determine  that  Neliis  AFB  was  best  suited  for 
this  particuiar  test. 


1 .  Fort  Fluachuca 

2.  Edwards  AFB 

3.  Kirtiand  AFB 


4.  Neilis  AFB 

5.  Eglin  AFB 

6.  Hanscom  AFB 


2 

2 

1 

3 

3 

Site 

1 

2 

3 

4 

5 

6 

4 

4 

4 

4 

Count 

2 

1 

1 

5 

4 

2 

5 

5 

5 

4 

5 

Rank 

3rd 

6th 

5th 

1st 

2nd 

4th 

1 

6 

6 

4 

5 
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PARETO  CHART 

PURPOSE:  To  help  identify  and  prioritize  issues  or  problems,  identify 

root  causes,  or  evaluate  results  of  improvement  areas.  The  Pareto 

Chart  graphically  displays  the  frequency  of  occurrence  of  data  items. 

METHOD: 

•  Decide  on  the  categories  of  items  (e.g.,  issues  or  causes)  on  which 
to  focus. 

•  Choose  the  measurement  units  which  provide  the  most  meaningful 
comparison  between  the  categories  of  items. 

•  Determine  the  time  period  to  collect  data. 

•  Collect  data  on  the  chosen  categories  of  items. 

•  Create  a  frequency  table  listing  all  the  categories,  the  frequency  of 
their  occurrence,  and  the  percentage  of  their  occurrence. 

•  Create  a  vertical  bar  chart  with  a  vertical  bar  (column)  for  each  of 
the  categories  you  measured  on  the  horizontal  axis,  starting  with  the 
category  with  the  highest  frequency  of  occurrence  on  the  far  left  side 
and  continuing  in  descending  order  to  the  right  to  the  category  with 
the  lowest  frequency  of  occurrence  on  the  far  right  side.  The  height 
of  the  bar  will  equal  the  frequency  (on  the  left  vertical  axis)  for  each 
of  the  categories. 

•  (Optional)  Draw  a  line  showing  the  cumulative  percentage  of  the 
categories  from  left  to  right  (0-100%).  Draw  a  vertical  axis  on  the 
right  side  showing  the  percentage  scale. 


Failure  Causes 
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BENCHMARKING 

Benchmarking  is  the  process  of  measuring  products,  services,  and 
practices  against  the  toughest  competitors  or  those  known  as  leaders 
in  their  field.  Benchmarking  can  help  you: 

•  Understand  how  you  compare  with  similar  organizations;  and 

•  Identify  areas  for  process  improvement. 

HOWTO  DO  IT: 

Identify  the  process  to  be  benchmarked.  Select  a  process  (as  op¬ 
posed  to  a  product)  that  is  important  to  both  your  organization  and  your 
customers.  Be  sure  the  process  in  your  organization  is  similar  to  and 
measured  in  the  same  manner  as  the  one  to  which  it’s  being  compared. 

Study  other  organizations.  Develop  a  list  of  organizations  with 
comparable  products  and  services.  Determine  what  specific  pro¬ 
cesses  the  organization  performs.  Based  on  this  information,  rank 
the  organizations  from  best  to  worst. 

Compare  and  evaluate.  Compare  your  process  to  the  best  and 
worst  cases  and  list  the  important  differences.  These  differences  can 
suggest  potential  improvements  to  your  process. 

BENCHMARKING  EXAMPLE: 

Using  inputs  their 
customers  provided, 
the  executive  leaders  at 
AF  Product  Division  B 
decided  that  their  source 
selection  process  needed 
improvement.  As  part  of 
the  initial  analysis,  they 
wanted  to  see  how  their 
process  compared  with 
others.  They  determined 
that  the  average  number 
of  days  required  for  source 
selection  was  an  important 
process  measure. 

As  a  result  of  this  analysis,  representatives  visited  AF  Product  Division 
A  and  Navy  Division  B  and  studied  their  source  selection  procedures. 

Note:  Benchmarking  is  nof  replicating  a  process  from  an  organization  that  excels  (unless 
your  goal  is  to  be  2nd  best).  It  is  studying  the  process,  clearly  understanding  the  theory 
behind  the  process,  and  then  restudying  your  own  process  to  determine  improvements. 


Cash 

received 


/ 


Accounts 

receivable 


\ 


Sale 

(DD250) 


Raw  material 
inventory 


tory 

I  \ 


Work  in  process 
inventory 


/ 

Finished  goods 
inventory 


Accounts 

payable 


Wages 

payable 

J 


Cash 

disbursed 


Cash 

disbursed 
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FLOWCHARTING 

PURPOSE:  To  identify  the  steps  or  tasks  in  a  process.  The  current 
process  can  then  be  analyzed  to  discover  duplicate  or  unnecessary 
actions,  bottlenecks,  or  other  problem  areas.  Ideas  for  improvement 
can  then  be  identified. 

METHOD: 

•  Clearly  define  where  the  process  begins  and  ends. 

•  List  all  of  the  steps  in  the  process,  including  decision  points,  and 
inputs  to  and  outputs  from  the  process. 

•  Arrange  the  steps  of  the  process  in  the  sequence  in  which  they 
currently  occur.  If  it  is  a  new  process,  begin  with  the  sequence  in 
which  you  believe  they  will  occur. 

•  Draw  the  appropriate  symbols  for  each  of  the  items  in  the  process. 

•  Label  the  items  in  the  process  with  text  describing  that  item. 

•  Add  arrows  showing  the  process  flow. 

•  Review  for  accuracy. 

—  Correct  symbols 

—  Correct  labels 

—  Correct  sequence 

—  Correct  direction  of  flow 


EXAMPLE: 


Common  Flowchart  Symbols 


(  ) 

Terminator  -  shows  the  begining  and  ending  points  of 
the  process.  Start  points  are  usually  some  sort  of  trigger 
acrivity  for  the  process. 

Activity  -  an  action  step  or  process  (within  the  process). 

o 

Decision  Point  -  where  a  decision  is  required;  usually 
with  two  options  (e.g.,  yes/no). 

Document  -  a  step  or  action  that  produces  a  document. 

o 

Connector  -  shows  a  jump  from  one  point  in  the  process 
to  another,  or  to  another  page.  Usually  labeled  with  letters 
(e.g.,  A<B<C<  etc.). 
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EXAMPLE: 
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DEPLOYMENT  FLOWCHARTS 

PURPOSE:  Depicts  a  process  and  the  individuals  or  teams  responsible 

for  the  steps/actions  in  the  process.  The  Deployment  Flowchart  can  be 

useful  to  clarify  individual  or  team  roles  and  responsibilities,  and  also 

to  detect/prevent  duplication  of  effort. 

METHOD: 

•  List  the  steps  of  the  current  process. 

•  Identify  the  individuals/teams  involved. 

•  Draw  the  Deployment  Flowchart  showing  the  activities,  decisions, 
inputs,  outputs,  documents,  etc.  (see  example  below). 

•  List  the  individuals/teams  across  the  top  of  the  chart,  and  the 
timeline  (if  applicable)  down  the  side  (see  example  below). 

•  Evaluate  the  current  process  for  possible  changes,  and  update  as 
necessary. 

EXAMPLE: 


(  End  ) 
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NOMINAL  GROUP  TECHNIQUE  (NGT) 

PURPOSE:  To  rank  or  prioritize  the  importance  of  issues,  alternatives, 

or  processes.  Helps  a  team  reach  consensus  quicker  by  showing  pre¬ 
liminary  areas  of  agreement.  Allows  individual  team  members  to  assign 

a  rank  or  priority  to  items  without  influence  or  pressure  from  others. 

METHOD: 

•  Brainstorm  a  list  of  the  issues,  alternatives,  or  processes  that  you 
are  analyzing. 

•  Compile  a  final  list  of  brainstorming  inputs  by  eliminating  duplicate 
or  similar  inputs,  and  clarifying  the  meanings  of  any  inputs  that  are 
unclear. 

•  Each  team  member  votes  by  ranking  the  inputs  in  order  of 
importance  (see  first  example  on  next  page). 

•  The  highest  number  is  generally  used  to  indicate  the  most  important 
or  highest  priority  item.  For  example,  if  team  members  are  ranking 

1 0  items,  “1 0”  would  represent  the  most  important  item  and  “1”  the 
least  important  item.  Make  sure  you  specify  the  values  used  in  the 
ranking,  i.e.,  which  number  represents  the  highest  or  most  important 
rating,  and  which  represents  the  lowest,  to  ensure  there  is  no 
confusion. 

•  Team  members  may  rank  all  of  the  items,  or  some  pre-designated 
portion  of  the  items  (particularly  when  there  is  a  long  list),  such  as  a 
third  or  a  half. 

•  Add  all  of  the  rankings,  and  analyze  the  results. 

•  Unless  the  team  is  pressed  for  time,  use  the  ranking  information  as  a 
starting  point  for  discussion  instead  of  accepting  it  as  a  “final  score.” 

•  An  alternate  method  is  to  assign  each  team  member  a  number  of 
points  (e.g.,  100),  which  they  allocate  across  the  options  (some  or 
all).  This  variation  is  known  as  weighted  multivoting  (see  second 
example  on  next  page). 

•  When  using  weighted  multivoting,  it’s  a  good  idea  to  assign  a 
maximum  number  of  points  that  can  be  assigned  to  any  one 
item  (e.g.,  40  out  of  100)  to  prevent  one  team  member  for  over¬ 
representing  the  relative  importance  of  an  item  (see  Item  H  in  the 
second  example  on  the  next  page). 
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Problems  with  Team  Performance  (Ranking  for  Example) 


Causes  (7) 

Peter 

Paul 

Mary 

Total 

Inadequate  team  training 

6 

3 

5 

14 

Unclear  objectives 

4 

7 

3 

14 

Insufficient  resource 

3 

5 

2 

10 

High  team  member  turnover 

2 

2 

4 

8 

Inefficient  team  process 

5 

6 

7 

18 

Team  member  hidden  agendas 

7 

4 

6 

17 

Poor  functional  representaion 

1 

1 

1 

3 

Weighted  Multivoting  Example 


Issues  (12) 

John 

Paul 

Ringo 

George 

Total 

Item  A 

40 

30 

10 

20 

100 

Item  B 

5 

5 

Item  C 

10 

10 

20 

40 

Item  D 

10 

20 

30 

Item  E 

20 

10 

30 

Item  F 

5 

5 

10 

Item  G 

15 

20 

20 

20 

75 

Item  H 

50 

50 

Item  1 

10 

5 

5 

20 

Item  J 

5 

10 

15 

Item  K 

10 

10 

Item  L 

5 

5 

5 

15 
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CREATIVE  PROBLEM  SOLVING 


✓  1 .  Identify  the  goal  or  challenge. 

\ 

y  2.  Gather  releveant  data. 

\ 

3.  Define  the  problem. 

^  4.  Generate  potential  solutions. 

\  5.  Select,  and  if  possible,  stengthen  the  solution. 

6.  Develop  a  plan  to  implement  the  chosen  solution. 

Each  six  steps  above  has  a  Divergent  Phase  (✓)  followed 
by  a  Convergent  Phase  (\ ).  See  below  for  a  description  of 
the  steps  in  each  phase 

DIVERGENT  PHASE 

1 .  Avoid  judging  or  evaluating  ideas  as  they  are  offered 

2.  Generate  as  many  ideas  as  possible 

3.  Accept  all  the  ideas  generated 

4.  Stretch  your  thinking 

5.  Allow  time  for  ideas  to  “grow” 

6.  Combine  or  connect  ideas  or  concepts 

CONVERGENT  PHASE 

1 .  Use  a  logical,  methodical  approach  to  make  choices 

2.  Clearly  and  specifically  state  the  basis  for  evaluating 

3.  Avoid  a  rush  to  closure 

4.  Don’t  ignore  or  avoid  difficult  issues  1/ 

5.  Look  for  strengths  or  positive  aspects  of  ideas 

6.  Remain  focused  on  the  objectives 


or  decisions 
ideas  ^  ^ 


Diverge 

Converge 
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RESPONSIBILITY  ASSIGNMENT  MATRIX  (RAM) 

PURPOSE:  Display  actions,  tasks,  or  assignments;  and  what  respon¬ 
sibilities  each  individual  has  for  them. 


METHOD: 

•  Brainstorm  a  list  of  actions  or  tasks  the  team  must  complete. 

•  List  the  team  members  and  other  stakeholders  who  may  have 
responsibilities. 

•  Construct  a  matrix  with  the  actions/tasks  listed  down  the  left  side  of 
the  matrix,  and  the  people  listed  across  the  top. 

•  Choose  the  symbols  to  indicate  the  level  of  responsibility 
represented  in  the  matrix  (e.g.,  primary,  secondary,  keep  informed, 
etc.). 

•  Agree  on  the  individual  responsibilities  and  complete  the  matrix  by 
placing  the  symbols  for  each  step/task  under  the  appropriate  people. 

•  Generally,  only  one  person  should  have  primary  responsibility,  others 
with  some  responsibility  would  have  secondary  responsibility. 


EXAMPLE: 


Goal: 

Establish  and 
maintain  an 
effective  risk 
management 
program 


Legend: 

0=  Primary 
Respond 

Q  =  Second 
Respond 

R  =  Review 

C  =  Coord 

I  =  Input 


o 

D) 

o 

o 

1- 

(A 

0. 

£ 

cc 

CE 

Communicate  Risk  Mgmt  Goals 

Identify  Risk  Coordinator 

Establish  Risk  Mgmt  IPT  (RM  IPT) 

® 

Draft  Risk  Mgmt  Plan 

® 

' 

Approve  Risk  Mgmt  Plan 

® 

R 

Identify  Risk  Events 

® 

C  1  1 

Analyze  Risks 

® 

C  1  1 

Prioritize  Risks 

® 

C  1  1 

ID/Evaluate  Mitigation  Strategies 

® 

R  1  1 

Select  Risk  Mitigation  Strategies 

® 

R 

C 

Enter  Risk  in  Risk  M13  (RM13) 

o 

® 
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KNOT  CHART 

The  KNOT  Chart  is  useful  for: 

•  Initially  sorting  the  wheat  from  the  chaff. 

•  Organizing/coordinating  the  next  steps  of  the  problem-solving 
process. 


Know 

Need  to  Know 

Opinion 

Think 

We  Know 

As  you  work  your  way  through  the  problem,  everything  should  move 
into  the  left  column — “Know.” 
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QUALITATIVE  PROBLEM  SOLVING 


Deviation  Statement:  Describe  the  actuai  performance  vs.  should  performance 


Is 

Is  Not 

What  Is  distinctive  about 
“Is”  vs.  “Is  Not”? 

Does  the  distinction 
suggest  a  change? 

Specifying 

Question 

What? 

(identify) 

Where? 

(Location) 

When? 

(Timing) 

Extent? 

(Magnitude) 

Possibie  Causes: 

Most  Likeiy  Cause: 

1.  Define  deviation. 

2.  Describe  what  deviation  IS  and  IS  NOT. 

3.  List  distinctions  between  what  deviation  IS  and  IS  NOT. 

4.  Do  distinctions  indicate  or  suggest  a  change? 

5.  Determine  possible  causes  based  on  distinctions  and  changes. 

Copyright  KepnerTregoe,  Inc.  (1981).  All  rights  reserved.  Reprinted  \A/ith  permission.  (Kepner-Tregoe, 
Inc.,  Research  Road,  P.O.  Box  704,  Princeton,  N.J.  08542) 
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GANTT  CHART 

A  Gantt  Chart  is  used  for  planning  schedules  and  managing  projects. 
It  is  a  method  for  basic  planning  and  work  instruction. 

How  to  do  it: 

1 .  The  Gantt  Process  begins  by  listing  the  activities  of  a  project  in 
order  of  execution. 


ACTIVITIES 

1 .  Requirements  are  written 

2.  Finances  are  arranged 

3.  Bidding  takes  place 

4.  Contractor  is  selected 

5.  Prototype  is  built 

6.  Testing  begins 


2.  Place  the  number  of  each  activity  across  the  top  of  your  chart. 
Time  duration  such  as  days,  weeks,  years,  etc.,  can  replace 
activity  numbers  if  appropriate. 

3.  Draw  vertical  lines  across  the  chart  for  each  item. 

4.  Starting  with  number  1,  begin  comparing  the  activities.  Can 
number  1  be  done  at  the  same  time  as  number  5  or  6? 

5.  Draw  horizontal  lines  to  indicate  which  activities  can  be  done 
simultaneously. 


ACTIVITIES 

1 

2 

3 

4 

5 

6 

1 .  Requirements  are  written 

2.  Finances  are  arranged 

3.  Bidding  takes  place 

4.  Contractor  is  selected 

5.  Prototype  is  built 

6.  Testing  begins 

6.  You  now  have  an  overview  of  your  project  giving  you  a  starting 
point  and  time-saving  measures  to  help  you  complete  the  project 
on  time. 
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